From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 11:13:23 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 11:13:23 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 11:13:23 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Jan 13 11:13:23 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 11:13:23 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 11:13:23 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Jan 13 11:13:23 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 11:13:23 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 11:13:23 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 11:13:23 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Jan 13 11:13:23 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 11:13:23 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 11:13:24 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 15:13:56 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 15:13:56 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 15:13:56 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Jan 13 15:13:56 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 15:13:56 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 15:13:56 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Jan 13 15:13:56 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 15:13:56 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 15:13:56 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 15:13:56 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Jan 13 15:13:56 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 15:13:56 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 15:13:56 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 15:21:18 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 15:21:18 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 15:21:18 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Jan 13 15:21:18 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 15:21:18 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 15:21:18 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Jan 13 15:21:18 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 15:21:18 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 15:21:18 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 15:21:18 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Jan 13 15:21:18 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 15:21:18 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 15:21:18 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 16:00:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 16:00:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 16:00:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Jan 13 16:00:51 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 16:00:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 16:00:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Jan 13 16:00:51 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 16:00:51 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 16:00:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 16:00:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Jan 13 16:00:51 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 16:00:51 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 16:00:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 20:01:46 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 20:01:46 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 20:01:46 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Jan 13 20:01:46 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 20:01:46 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 20:01:46 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Jan 13 20:01:46 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 20:01:46 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 13 20:01:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 20:01:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Jan 13 20:01:47 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 20:01:47 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 13 20:01:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 08:02:24 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 08:02:24 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 08:02:24 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Jan 14 08:02:24 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 08:02:24 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 08:02:24 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Jan 14 08:02:24 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 08:02:24 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 08:02:24 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 08:02:24 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Jan 14 08:02:24 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 08:02:24 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 08:02:24 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 12:00:50 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 12:00:50 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 12:00:50 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Jan 14 12:00:50 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 12:00:50 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 12:00:50 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Jan 14 12:00:50 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 12:00:50 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 12:00:50 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 12:00:50 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Jan 14 12:00:50 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 12:00:50 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 12:00:50 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 16:00:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 16:00:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 16:00:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Jan 14 16:00:47 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 16:00:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 16:00:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Jan 14 16:00:47 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 16:00:47 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 16:00:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 16:00:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Jan 14 16:00:47 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 16:00:47 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 16:00:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 20:00:46 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 20:00:46 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 20:00:46 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Jan 14 20:00:46 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 20:00:46 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 20:00:46 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Jan 14 20:00:46 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 20:00:46 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 14 20:00:46 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 20:00:46 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Jan 14 20:00:46 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 20:00:46 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 14 20:00:46 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 08:02:13 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 08:02:13 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 08:02:13 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Jan 15 08:02:13 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 08:02:13 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 08:02:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Jan 15 08:02:13 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 08:02:13 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 08:02:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 08:02:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Jan 15 08:02:13 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 08:02:13 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 08:02:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 12:00:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 12:00:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 12:00:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Jan 15 12:00:55 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 12:00:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 12:00:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Jan 15 12:00:55 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 12:00:55 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 12:00:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 12:00:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Jan 15 12:00:55 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 12:00:55 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 12:00:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 16:00:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 16:00:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 16:00:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Jan 15 16:00:47 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 16:00:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 16:00:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Jan 15 16:00:47 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 16:00:47 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 16:00:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 16:00:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Jan 15 16:00:47 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 16:00:47 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 16:00:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 20:00:46 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 20:00:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 20:00:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Jan 15 20:00:47 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 20:00:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 20:00:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Jan 15 20:00:47 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 20:00:47 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 15 20:00:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 20:00:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Jan 15 20:00:47 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 20:00:47 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 15 20:00:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 08:02:35 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 08:02:35 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 08:02:35 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Jan 16 08:02:35 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 08:02:35 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 08:02:35 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Jan 16 08:02:35 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 08:02:35 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 08:02:35 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 08:02:35 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Jan 16 08:02:35 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 08:02:35 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 08:02:35 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 12:00:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 12:00:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 12:00:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Jan 16 12:00:51 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 12:00:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 12:00:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Jan 16 12:00:51 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 12:00:51 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 12:00:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 12:00:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Jan 16 12:00:51 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 12:00:51 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 12:00:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 16:00:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 16:00:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 16:00:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Jan 16 16:00:51 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 16:00:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 16:00:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Jan 16 16:00:51 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 16:00:51 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 16:00:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 16:00:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Jan 16 16:00:51 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 16:00:51 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 16:00:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 20:00:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 20:00:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 20:00:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Jan 16 20:00:47 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 20:00:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 20:00:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Jan 16 20:00:47 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 20:00:47 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 16 20:00:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 20:00:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Jan 16 20:00:47 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 20:00:47 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 16 20:00:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 08:02:37 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 08:02:37 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 08:02:38 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Jan 17 08:02:38 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 08:02:38 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 08:02:38 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Jan 17 08:02:38 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 08:02:38 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 08:02:38 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 08:02:38 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Jan 17 08:02:38 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 08:02:38 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 08:02:38 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 12:01:05 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 12:01:07 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 12:01:07 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Jan 17 12:01:07 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 12:01:07 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 12:01:07 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Jan 17 12:01:07 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 12:01:07 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 12:01:07 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 12:01:07 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Jan 17 12:01:07 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 12:01:07 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 12:01:07 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 16:00:53 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 16:00:53 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 16:00:53 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Jan 17 16:00:53 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 16:00:53 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 16:00:53 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Jan 17 16:00:53 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 16:00:53 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 16:00:53 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 16:00:53 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Jan 17 16:00:53 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 16:00:53 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 16:00:53 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 20:00:54 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 20:00:54 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 20:00:54 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Jan 17 20:00:54 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 20:00:54 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 20:00:54 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Jan 17 20:00:54 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 20:00:54 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 17 20:00:54 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 20:00:54 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Jan 17 20:00:54 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 20:00:54 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 17 20:00:54 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 08:04:22 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 08:04:22 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 08:04:22 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Jan 18 08:04:22 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 08:04:22 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 08:04:22 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Jan 18 08:04:22 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 08:04:22 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 08:04:22 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 08:04:22 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Jan 18 08:04:22 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 08:04:22 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 08:04:22 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 12:01:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 12:01:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 12:01:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Jan 18 12:01:09 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 12:01:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 12:01:09 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Jan 18 12:01:09 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 12:01:09 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 12:01:09 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 12:01:09 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Jan 18 12:01:09 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 12:01:09 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 12:01:09 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 16:01:34 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 16:01:34 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 16:01:34 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Jan 18 16:01:34 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 16:01:34 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 16:01:34 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Jan 18 16:01:34 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 16:01:34 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 16:01:34 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 16:01:34 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Jan 18 16:01:34 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 16:01:35 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 16:01:35 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 20:00:58 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 20:00:58 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 20:00:58 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Jan 18 20:00:58 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 20:00:58 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 20:00:58 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Jan 18 20:00:58 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 20:00:58 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 18 20:00:58 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 20:00:58 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Jan 18 20:00:59 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 20:00:59 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 18 20:00:59 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 08:04:45 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 08:04:45 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 08:04:45 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Thu Jan 19 08:04:45 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 08:04:45 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 08:04:45 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Thu Jan 19 08:04:45 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 08:04:45 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 08:04:45 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 08:04:45 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Thu Jan 19 08:04:45 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 08:04:45 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 08:04:45 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 12:01:03 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 12:01:03 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 12:01:03 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Thu Jan 19 12:01:03 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 12:01:03 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 12:01:03 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Thu Jan 19 12:01:03 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 12:01:03 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 12:01:03 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 12:01:03 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Thu Jan 19 12:01:03 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 12:01:03 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 12:01:03 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 16:03:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 16:03:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 16:03:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Thu Jan 19 16:03:11 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 16:03:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 16:03:12 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Thu Jan 19 16:03:12 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 16:03:12 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 16:03:12 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 16:03:12 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Thu Jan 19 16:03:12 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 16:03:12 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 16:03:12 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 20:00:52 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 20:00:52 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 20:00:52 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Thu Jan 19 20:00:52 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 20:00:52 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 20:00:52 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Thu Jan 19 20:00:52 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 20:00:52 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 19 20:00:52 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 20:00:52 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Thu Jan 19 20:00:52 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 20:00:52 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 19 20:00:52 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 08:07:22 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 08:07:22 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 08:07:22 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Jan 20 08:07:22 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 08:07:22 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 08:07:22 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Jan 20 08:07:22 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 08:07:22 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 08:07:22 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 08:07:22 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Jan 20 08:07:22 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 08:07:22 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 08:07:22 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 12:02:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 12:02:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 12:02:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Jan 20 12:02:11 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 12:02:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 12:02:11 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Jan 20 12:02:11 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 12:02:11 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 12:02:11 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 12:02:11 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Jan 20 12:02:11 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 12:02:11 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 12:02:11 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 16:00:56 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 16:00:56 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 16:00:56 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Jan 20 16:00:56 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 16:00:56 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 16:00:56 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Jan 20 16:00:56 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 16:00:56 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 16:00:56 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 16:00:56 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Jan 20 16:00:56 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 16:00:56 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 16:00:56 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 20:00:50 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 20:00:50 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 20:00:50 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Jan 20 20:00:50 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 20:00:50 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 20:00:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Jan 20 20:00:51 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 20:00:51 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 20 20:00:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 20:00:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Jan 20 20:00:51 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 20:00:51 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 20 20:00:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 08:03:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 08:03:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 08:03:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Jan 21 08:03:59 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 08:03:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 08:03:59 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Jan 21 08:03:59 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 08:03:59 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 08:03:59 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 08:03:59 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Jan 21 08:03:59 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 08:03:59 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 08:03:59 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 12:01:05 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 12:01:05 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 12:01:05 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Jan 21 12:01:05 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 12:01:05 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 12:01:05 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Jan 21 12:01:05 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 12:01:05 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 12:01:05 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 12:01:05 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Jan 21 12:01:05 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 12:01:05 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 12:01:05 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 16:00:54 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 16:00:54 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 16:00:54 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Jan 21 16:00:54 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 16:00:54 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 16:00:54 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Jan 21 16:00:54 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 16:00:54 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 16:00:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 16:00:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Jan 21 16:00:55 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 16:00:55 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 16:00:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 20:00:53 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 20:00:53 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 20:00:53 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Jan 21 20:00:53 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 20:00:53 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 20:00:53 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Jan 21 20:00:53 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 20:00:53 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 21 20:00:53 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 20:00:53 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Jan 21 20:00:53 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 20:00:53 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 21 20:00:53 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 08:02:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 08:02:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 08:02:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Jan 22 08:02:51 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 08:02:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 08:02:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Jan 22 08:02:51 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 08:02:51 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 08:02:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 08:02:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Jan 22 08:02:51 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 08:02:51 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 08:02:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 12:00:54 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 12:00:54 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 12:00:54 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Jan 22 12:00:54 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 12:00:54 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 12:00:54 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Jan 22 12:00:54 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 12:00:54 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 12:00:54 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 12:00:54 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Jan 22 12:00:55 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 12:00:55 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 12:00:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 16:01:26 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 16:01:26 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 16:01:26 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Jan 22 16:01:26 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 16:01:26 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 16:01:26 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Jan 22 16:01:26 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 16:01:26 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 16:01:26 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 16:01:26 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Jan 22 16:01:26 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 16:01:26 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 16:01:26 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 20:01:21 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 20:01:21 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 20:01:21 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Jan 22 20:01:21 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 20:01:21 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 20:01:21 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Jan 22 20:01:21 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 20:01:21 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 22 20:01:21 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 20:01:21 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Jan 22 20:01:21 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 20:01:21 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 22 20:01:21 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 08:02:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 08:02:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 08:02:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Jan 23 08:02:47 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 08:02:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 08:02:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Jan 23 08:02:47 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 08:02:47 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 08:02:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 08:02:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Jan 23 08:02:47 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 08:02:47 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 08:02:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 12:00:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 12:00:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 12:00:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Jan 23 12:00:59 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 12:00:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 12:00:59 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Jan 23 12:00:59 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 12:00:59 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 12:00:59 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 12:00:59 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Jan 23 12:00:59 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 12:00:59 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 12:00:59 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 16:01:54 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 16:01:54 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 16:01:54 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Jan 23 16:01:55 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 16:01:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 16:01:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Jan 23 16:01:55 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 16:01:55 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 16:01:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 16:01:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Jan 23 16:01:55 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 16:01:55 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 16:01:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 20:00:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 20:00:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 20:00:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Jan 23 20:00:55 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 20:00:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 20:00:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Jan 23 20:00:55 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 20:00:55 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 23 20:00:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 20:00:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Jan 23 20:00:55 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 20:00:55 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 23 20:00:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 08:03:20 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 08:03:20 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 08:03:20 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Jan 24 08:03:20 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 08:03:20 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 08:03:20 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Jan 24 08:03:20 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 08:03:20 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 08:03:20 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 08:03:20 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Jan 24 08:03:20 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 08:03:20 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 08:03:20 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 12:03:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 12:03:12 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 12:03:12 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Jan 24 12:03:12 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 12:03:12 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 12:03:12 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Jan 24 12:03:12 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 12:03:12 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 12:03:12 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 12:03:12 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Jan 24 12:03:12 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 12:03:13 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 12:03:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 16:01:29 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 16:01:30 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 16:01:30 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Jan 24 16:01:30 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 16:01:30 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 16:01:30 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Jan 24 16:01:30 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 16:01:30 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 16:01:30 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 16:01:30 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Jan 24 16:01:30 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 16:01:30 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 16:01:30 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 20:02:00 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 20:02:00 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 20:02:00 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Jan 24 20:02:00 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 20:02:00 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 20:02:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Jan 24 20:02:00 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 20:02:00 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 24 20:02:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 20:02:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Jan 24 20:02:00 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 20:02:00 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 24 20:02:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 08:02:08 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 08:02:08 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 08:02:08 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Jan 25 08:02:08 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 08:02:08 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 08:02:08 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Jan 25 08:02:08 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 08:02:08 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 08:02:08 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 08:02:08 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Jan 25 08:02:08 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 08:02:08 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 08:02:08 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 12:01:08 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 12:01:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 12:01:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Jan 25 12:01:09 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 12:01:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 12:01:09 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Jan 25 12:01:09 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 12:01:09 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 12:01:09 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 12:01:09 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Jan 25 12:01:09 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 12:01:09 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 12:01:09 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 16:01:03 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 16:01:03 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 16:01:03 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Jan 25 16:01:03 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 16:01:03 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 16:01:03 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Jan 25 16:01:03 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 16:01:03 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 16:01:03 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 16:01:03 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Jan 25 16:01:03 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 16:01:04 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 16:01:04 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 20:00:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 20:00:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 20:00:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Jan 25 20:00:59 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 20:00:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 20:00:59 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Jan 25 20:00:59 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 20:00:59 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jan 25 20:01:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 20:01:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Jan 25 20:01:00 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 20:01:00 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Jan 25 20:01:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 08:06:01 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 08:06:01 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 08:06:01 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Thu Jan 26 08:06:01 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 08:06:01 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 08:06:01 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Thu Jan 26 08:06:01 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 08:06:01 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 08:06:01 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 08:06:01 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Thu Jan 26 08:06:01 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 08:06:01 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 08:06:01 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 12:01:10 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 12:01:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 12:01:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Thu Jan 26 12:01:11 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 12:01:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 12:01:11 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Thu Jan 26 12:01:11 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 12:01:11 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 12:01:11 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 12:01:11 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Thu Jan 26 12:01:11 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 12:01:11 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 12:01:11 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 16:00:57 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 16:00:57 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 16:00:57 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Thu Jan 26 16:00:57 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 16:00:57 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 16:00:57 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Thu Jan 26 16:00:57 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 16:00:57 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 16:00:57 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 16:00:57 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Thu Jan 26 16:00:57 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 16:00:57 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 16:00:57 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 20:00:50 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 20:00:50 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 20:00:50 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Thu Jan 26 20:00:50 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 20:00:50 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 20:00:50 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Thu Jan 26 20:00:50 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 20:00:50 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 26 20:00:50 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 20:00:50 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Thu Jan 26 20:00:50 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 20:00:50 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Jan 26 20:00:50 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 08:06:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 08:06:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 08:06:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Jan 27 08:06:55 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 08:06:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 08:06:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Jan 27 08:06:55 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 08:06:55 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 08:06:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 08:06:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Jan 27 08:06:55 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 08:06:55 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 08:06:56 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 12:02:17 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 12:02:17 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 12:02:17 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Jan 27 12:02:17 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 12:02:17 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 12:02:17 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Jan 27 12:02:17 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 12:02:17 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 12:02:17 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 12:02:17 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Jan 27 12:02:17 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 12:02:17 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 12:02:17 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 16:01:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 16:01:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 16:01:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Jan 27 16:01:11 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 16:01:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 16:01:11 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Jan 27 16:01:11 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 16:01:11 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 16:01:11 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 16:01:11 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Jan 27 16:01:11 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 16:01:11 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 16:01:11 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 20:01:02 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 20:01:02 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 20:01:02 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Jan 27 20:01:02 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 20:01:02 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 20:01:02 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Jan 27 20:01:02 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 20:01:02 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 27 20:01:02 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 20:01:02 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Jan 27 20:01:02 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 20:01:02 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Jan 27 20:01:02 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 08:05:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 08:05:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 08:05:52 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Jan 28 08:05:52 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 08:05:52 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 08:05:52 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Jan 28 08:05:52 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 08:05:52 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 08:05:52 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 08:05:52 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Jan 28 08:05:52 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 08:05:52 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 08:05:52 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 12:01:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 12:02:00 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 12:02:00 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Jan 28 12:02:00 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 12:02:00 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 12:02:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Jan 28 12:02:00 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 12:02:00 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 12:02:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 12:02:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Jan 28 12:02:00 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 12:02:00 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 12:02:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 16:00:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 16:00:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 16:00:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Jan 28 16:00:59 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 16:00:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 16:01:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Jan 28 16:01:00 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 16:01:00 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 16:01:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 16:01:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Jan 28 16:01:00 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 16:01:00 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 16:01:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 20:01:00 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 20:01:00 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 20:01:00 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Jan 28 20:01:00 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 20:01:00 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 20:01:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Jan 28 20:01:00 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 20:01:00 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Jan 28 20:01:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 20:01:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Jan 28 20:01:00 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 20:01:00 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Jan 28 20:01:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 08:07:39 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 08:07:39 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 08:07:39 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Jan 29 08:07:39 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 08:07:39 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 08:07:39 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Jan 29 08:07:39 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 08:07:39 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 08:07:39 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 08:07:39 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Jan 29 08:07:40 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 08:07:40 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 08:07:40 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 12:01:05 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 12:01:05 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 12:01:05 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Jan 29 12:01:05 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 12:01:05 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 12:01:05 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Jan 29 12:01:05 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 12:01:06 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 12:01:06 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 12:01:06 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Jan 29 12:01:06 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 12:01:06 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 12:01:06 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 16:00:58 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 16:00:58 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 16:00:58 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Jan 29 16:00:58 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 16:00:58 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 16:00:58 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Jan 29 16:00:58 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 16:00:58 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 16:00:58 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 16:00:58 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Jan 29 16:00:58 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 16:00:58 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 16:00:58 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 20:01:00 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 20:01:00 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 20:01:00 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Jan 29 20:01:00 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 20:01:00 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 20:01:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Jan 29 20:01:00 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 20:01:00 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Jan 29 20:01:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 20:01:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Jan 29 20:01:00 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 20:01:00 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Jan 29 20:01:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 08:09:16 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 08:09:17 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 08:09:17 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Jan 30 08:09:18 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 08:09:18 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 08:09:18 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Jan 30 08:09:18 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 08:09:18 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 08:09:18 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 08:09:18 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Jan 30 08:09:18 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 08:09:18 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 08:09:19 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 12:04:03 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 12:04:03 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 12:04:03 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Jan 30 12:04:03 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 12:04:03 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 12:04:03 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Jan 30 12:04:03 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 12:04:03 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 12:04:03 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 12:04:03 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Jan 30 12:04:03 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 12:04:03 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 12:04:03 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 16:04:20 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 16:04:20 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 16:04:20 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Jan 30 16:04:20 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 16:04:20 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 16:04:20 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Jan 30 16:04:20 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 16:04:20 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 16:04:20 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 16:04:20 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Jan 30 16:04:20 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 16:04:20 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 16:04:20 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 20:02:19 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 20:02:20 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 20:02:20 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Jan 30 20:02:20 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 20:02:20 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 20:02:20 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Jan 30 20:02:20 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 20:02:20 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 30 20:02:20 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 20:02:20 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Jan 30 20:02:20 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 20:02:20 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Jan 30 20:02:20 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 08:08:02 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 08:08:03 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 08:08:03 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Jan 31 08:08:03 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 08:08:03 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 08:08:03 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Jan 31 08:08:03 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 08:08:03 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 08:08:03 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 08:08:03 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Jan 31 08:08:03 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 08:08:03 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 08:08:03 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 12:04:25 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 12:04:25 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 12:04:25 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Jan 31 12:04:25 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 12:04:25 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 12:04:25 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Jan 31 12:04:25 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 12:04:25 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 12:04:25 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 12:04:25 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Jan 31 12:04:25 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 12:04:25 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 12:04:25 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 16:02:27 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 16:02:27 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 16:02:27 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Jan 31 16:02:27 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 16:02:27 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 16:02:28 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Jan 31 16:02:28 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 16:02:28 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 16:02:28 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 16:02:28 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Jan 31 16:02:28 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 16:02:28 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 16:02:28 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 20:01:13 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 20:01:13 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 20:01:13 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Jan 31 20:01:13 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 20:01:13 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 20:01:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Jan 31 20:01:13 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 20:01:13 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 31 20:01:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 20:01:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Jan 31 20:01:13 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 20:01:13 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Jan 31 20:01:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 08:10:07 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 08:10:07 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 08:10:08 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Feb 1 08:10:08 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 08:10:08 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 08:10:08 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Feb 1 08:10:08 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 08:10:08 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 08:10:08 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 08:10:08 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Feb 1 08:10:08 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 08:10:08 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 08:10:08 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 12:02:50 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 12:02:50 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 12:02:50 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Feb 1 12:02:50 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 12:02:50 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 12:02:50 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Feb 1 12:02:50 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 12:02:50 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 12:02:50 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 12:02:50 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Feb 1 12:02:50 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 12:02:50 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 12:02:50 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 16:01:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 16:01:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 16:01:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Feb 1 16:01:11 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 16:01:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 16:01:11 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Feb 1 16:01:11 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 16:01:13 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 16:01:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 16:01:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Feb 1 16:01:13 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 16:01:13 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 16:01:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 20:01:07 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 20:01:07 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 20:01:07 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Feb 1 20:01:07 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 20:01:07 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 20:01:07 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Feb 1 20:01:07 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 20:01:07 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 1 20:01:07 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 20:01:07 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Feb 1 20:01:07 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 20:01:07 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 1 20:01:07 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 08:13:21 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 08:13:21 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 08:13:22 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Thu Feb 2 08:13:22 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 08:13:22 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 08:13:22 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Thu Feb 2 08:13:22 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 08:13:22 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 08:13:22 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 08:13:22 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Thu Feb 2 08:13:22 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 08:13:22 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 08:13:22 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 12:02:57 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 12:02:57 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 12:02:57 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Thu Feb 2 12:02:57 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 12:02:57 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 12:02:57 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Thu Feb 2 12:02:57 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 12:02:57 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 12:02:57 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 12:02:57 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Thu Feb 2 12:02:57 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 12:02:57 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 12:02:57 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 16:01:16 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 16:01:16 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 16:01:16 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Thu Feb 2 16:01:16 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 16:01:16 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 16:01:16 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Thu Feb 2 16:01:16 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 16:01:17 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 16:01:17 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 16:01:17 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Thu Feb 2 16:01:17 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 16:01:17 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 16:01:17 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 20:01:06 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 20:01:06 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 20:01:06 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Thu Feb 2 20:01:06 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 20:01:06 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 20:01:06 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Thu Feb 2 20:01:06 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 20:01:06 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 2 20:01:06 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 20:01:07 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Thu Feb 2 20:01:07 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 20:01:07 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 2 20:01:07 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 08:12:33 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 08:12:33 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 08:12:33 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Feb 3 08:12:33 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 08:12:33 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 08:12:33 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Feb 3 08:12:33 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 08:12:33 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 08:12:34 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 08:12:34 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Feb 3 08:12:34 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 08:12:34 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 08:12:34 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 12:02:49 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 12:02:49 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 12:02:49 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Feb 3 12:02:49 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 12:02:49 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 12:02:49 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Feb 3 12:02:49 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 12:02:49 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 12:02:49 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 12:02:49 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Feb 3 12:02:49 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 12:02:49 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 12:02:49 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 16:01:06 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 16:01:06 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 16:01:07 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Feb 3 16:01:07 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 16:01:07 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 16:01:07 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Feb 3 16:01:07 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 16:01:07 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 16:01:07 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 16:01:07 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Feb 3 16:01:07 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 16:01:07 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 16:01:07 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 20:01:13 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 20:01:13 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 20:01:13 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Feb 3 20:01:13 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 20:01:13 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 20:01:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Feb 3 20:01:13 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 20:01:13 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 3 20:01:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 20:01:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Feb 3 20:01:13 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 20:01:13 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 3 20:01:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 08:08:19 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 08:08:19 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 08:08:19 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Feb 4 08:08:19 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 08:08:19 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 08:08:19 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Feb 4 08:08:19 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 08:08:19 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 08:08:19 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 08:08:19 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Feb 4 08:08:20 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 08:08:20 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 08:08:20 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 12:02:49 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 12:02:49 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 12:02:49 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Feb 4 12:02:49 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 12:02:49 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 12:02:49 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Feb 4 12:02:50 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 12:02:50 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 12:02:50 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 12:02:50 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Feb 4 12:02:50 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 12:02:50 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 12:02:50 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 16:01:18 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 16:01:18 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 16:01:18 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Feb 4 16:01:18 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 16:01:18 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 16:01:18 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Feb 4 16:01:18 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 16:01:18 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 16:01:18 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 16:01:18 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Feb 4 16:01:18 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 16:01:18 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 16:01:18 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 20:01:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 20:01:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 20:01:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Feb 4 20:01:09 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 20:01:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 20:01:09 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Feb 4 20:01:09 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 20:01:09 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 4 20:01:09 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 20:01:09 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Feb 4 20:01:09 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 20:01:09 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 4 20:01:09 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 08:08:03 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 08:08:04 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 08:08:04 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Feb 5 08:08:04 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 08:08:04 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 08:08:04 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Feb 5 08:08:04 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 08:08:04 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 08:08:04 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 08:08:04 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Feb 5 08:08:04 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 08:08:04 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 08:08:04 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 12:02:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 12:02:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 12:02:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Feb 5 12:02:51 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 12:02:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 12:02:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Feb 5 12:02:51 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 12:02:51 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 12:02:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 12:02:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Feb 5 12:02:51 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 12:02:51 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 12:02:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 16:01:14 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 16:01:14 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 16:01:14 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Feb 5 16:01:14 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 16:01:14 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 16:01:14 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Feb 5 16:01:14 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 16:01:14 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 16:01:14 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 16:01:14 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Feb 5 16:01:14 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 16:01:14 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 16:01:14 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 20:01:04 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 20:01:04 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 20:01:04 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Feb 5 20:01:04 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 20:01:04 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 20:01:04 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Feb 5 20:01:04 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 20:01:04 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 5 20:01:04 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 20:01:04 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Feb 5 20:01:04 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 20:01:04 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 5 20:01:04 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 08:15:50 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 08:15:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 08:15:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Feb 6 08:15:51 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 08:15:51 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 08:15:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Feb 6 08:15:51 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 08:15:51 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 08:15:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 08:15:51 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Feb 6 08:15:51 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 08:15:51 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 08:15:52 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 12:02:49 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 12:02:49 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 12:02:49 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Feb 6 12:02:49 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 12:02:49 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 12:02:49 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Feb 6 12:02:49 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 12:02:49 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 12:02:49 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 12:02:49 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Feb 6 12:02:49 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 12:02:49 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 12:02:49 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 16:01:10 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 16:01:10 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 16:01:10 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Feb 6 16:01:10 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 16:01:10 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 16:01:10 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Feb 6 16:01:10 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 16:01:10 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 16:01:10 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 16:01:10 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Feb 6 16:01:10 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 16:01:10 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 16:01:10 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 20:01:05 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 20:01:05 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 20:01:05 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Feb 6 20:01:05 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 20:01:05 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 20:01:05 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Feb 6 20:01:05 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 20:01:05 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 6 20:01:05 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 20:01:05 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Feb 6 20:01:05 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 20:01:05 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 6 20:01:05 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 08:24:36 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 08:24:36 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 08:24:37 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Feb 7 08:24:37 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 08:24:37 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 08:24:37 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Feb 7 08:24:37 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 08:24:37 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 08:24:37 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 08:24:37 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Feb 7 08:24:37 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 08:24:37 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 08:24:37 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 12:03:29 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 12:03:30 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 12:03:30 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Feb 7 12:03:30 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 12:03:30 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 12:03:30 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Feb 7 12:03:30 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 12:03:30 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 12:03:30 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 12:03:30 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Feb 7 12:03:30 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 12:03:30 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 12:03:30 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 16:01:14 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 16:01:14 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 16:01:14 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Feb 7 16:01:14 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 16:01:14 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 16:01:14 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Feb 7 16:01:14 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 16:01:14 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 16:01:14 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 16:01:14 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Feb 7 16:01:14 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 16:01:14 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 16:01:14 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 20:01:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 20:01:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 20:01:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Feb 7 20:01:09 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 20:01:10 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 20:01:10 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Feb 7 20:01:10 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 20:01:10 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 7 20:01:10 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 20:01:10 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Feb 7 20:01:10 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 20:01:10 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 7 20:01:10 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 08:23:28 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 08:23:29 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 08:23:29 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Feb 8 08:23:29 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 08:23:29 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 08:23:29 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Feb 8 08:23:29 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 08:23:29 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 08:23:29 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 08:23:29 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Feb 8 08:23:29 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 08:23:29 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 08:23:29 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 12:03:31 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 12:03:31 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 12:03:31 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Feb 8 12:03:31 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 12:03:31 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 12:03:31 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Feb 8 12:03:31 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 12:03:31 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 12:03:31 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 12:03:32 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Feb 8 12:03:32 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 12:03:32 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 12:03:32 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 16:01:23 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 16:01:23 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 16:01:23 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Feb 8 16:01:23 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 16:01:23 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 16:01:23 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Feb 8 16:01:23 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 16:01:23 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 16:01:23 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 16:01:23 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Feb 8 16:01:23 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 16:01:23 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 16:01:23 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 20:01:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 20:01:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 20:01:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Feb 8 20:01:09 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 20:01:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 20:01:10 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Feb 8 20:01:10 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 20:01:10 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 8 20:01:10 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 20:01:10 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Feb 8 20:01:10 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 20:01:10 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 8 20:01:10 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 08:09:41 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 08:09:42 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 08:09:42 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Thu Feb 9 08:09:42 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 08:09:42 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 08:09:42 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Thu Feb 9 08:09:42 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 08:09:42 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 08:09:42 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 08:09:42 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Thu Feb 9 08:09:42 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 08:09:43 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 08:09:43 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 12:03:25 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 12:03:25 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 12:03:25 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Thu Feb 9 12:03:25 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 12:03:25 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 12:03:25 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Thu Feb 9 12:03:25 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 12:03:25 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 12:03:25 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 12:03:25 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Thu Feb 9 12:03:25 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 12:03:25 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 12:03:25 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 16:01:19 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 16:01:19 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 16:01:19 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Thu Feb 9 16:01:19 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 16:01:19 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 16:01:19 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Thu Feb 9 16:01:19 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 16:01:19 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 16:01:19 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 16:01:19 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Thu Feb 9 16:01:19 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 16:01:19 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 16:01:19 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 20:01:17 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 20:01:17 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 20:01:17 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Thu Feb 9 20:01:17 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 20:01:17 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 20:01:17 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Thu Feb 9 20:01:17 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 20:01:17 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 9 20:01:17 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 20:01:17 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Thu Feb 9 20:01:17 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 20:01:17 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 9 20:01:17 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 08:21:01 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 08:21:01 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 08:21:01 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Feb 10 08:21:01 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 08:21:03 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 08:21:03 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Feb 10 08:21:03 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 08:21:03 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 08:21:03 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 08:21:03 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Feb 10 08:21:03 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 08:21:03 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 08:21:05 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 12:03:46 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 12:03:46 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 12:03:46 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Feb 10 12:03:46 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 12:03:46 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 12:03:46 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Feb 10 12:03:46 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 12:03:46 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 12:03:46 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 12:03:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Feb 10 12:03:47 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 12:03:47 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 12:03:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 16:01:15 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 16:01:15 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 16:01:15 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Feb 10 16:01:15 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 16:01:15 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 16:01:15 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Feb 10 16:01:15 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 16:01:15 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 16:01:15 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 16:01:15 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Feb 10 16:01:15 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 16:01:15 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 16:01:15 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 20:01:17 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 20:01:17 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 20:01:17 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Feb 10 20:01:17 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 20:01:17 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 20:01:17 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Feb 10 20:01:17 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 20:01:18 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 10 20:01:18 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 20:01:18 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Feb 10 20:01:18 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 20:01:18 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Feb 10 20:01:18 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 08:13:09 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 08:13:10 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 08:13:10 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Feb 11 08:13:10 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 08:13:10 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 08:13:10 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Feb 11 08:13:10 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 08:13:10 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 08:13:10 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 08:13:10 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Feb 11 08:13:10 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 08:13:11 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 08:13:11 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 12:02:44 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 12:02:46 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 12:02:46 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Feb 11 12:02:46 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 12:02:46 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 12:02:46 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Feb 11 12:02:46 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 12:02:46 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 12:02:46 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 12:02:46 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Feb 11 12:02:46 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 12:02:46 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 12:02:46 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 16:01:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 16:01:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 16:01:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Feb 11 16:01:11 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 16:01:11 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 16:01:11 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Feb 11 16:01:11 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 16:01:11 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 16:01:11 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 16:01:11 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Feb 11 16:01:11 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 16:01:11 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 16:01:11 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 20:01:02 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 20:01:02 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 20:01:02 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sat Feb 11 20:01:02 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 20:01:02 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 20:01:02 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sat Feb 11 20:01:02 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 20:01:02 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Feb 11 20:01:02 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 20:01:02 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sat Feb 11 20:01:02 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 20:01:02 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sat Feb 11 20:01:02 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 08:09:43 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 08:09:43 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 08:09:43 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Feb 12 08:09:43 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 08:09:43 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 08:09:43 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Feb 12 08:09:43 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 08:09:43 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 08:09:43 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 08:09:43 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Feb 12 08:09:43 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 08:09:43 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 08:09:44 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 12:02:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 12:02:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 12:02:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Feb 12 12:02:55 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 12:02:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 12:02:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Feb 12 12:02:55 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 12:02:55 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 12:02:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 12:02:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Feb 12 12:02:55 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 12:02:55 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 12:02:55 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 16:01:15 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 16:01:15 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 16:01:15 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Feb 12 16:01:15 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 16:01:15 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 16:01:15 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Feb 12 16:01:15 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 16:01:15 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 16:01:15 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 16:01:15 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Feb 12 16:01:15 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 16:01:15 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 16:01:15 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 20:01:08 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 20:01:08 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 20:01:08 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Sun Feb 12 20:01:08 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 20:01:08 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 20:01:08 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Sun Feb 12 20:01:08 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 20:01:08 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Sun Feb 12 20:01:08 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 20:01:08 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Sun Feb 12 20:01:08 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 20:01:08 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Sun Feb 12 20:01:08 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 08:17:37 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 08:17:37 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 08:17:37 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Feb 13 08:17:37 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 08:17:37 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 08:17:37 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Feb 13 08:17:37 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 08:17:37 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 08:17:38 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 08:17:38 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Feb 13 08:17:38 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 08:17:38 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 08:17:38 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 12:03:15 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 12:03:15 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 12:03:15 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Feb 13 12:03:15 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 12:03:15 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 12:03:16 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Feb 13 12:03:16 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 12:03:16 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 12:03:16 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 12:03:16 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Feb 13 12:03:16 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 12:03:16 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 12:03:16 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 16:01:31 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 16:01:31 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 16:01:31 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Feb 13 16:01:31 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 16:01:31 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 16:01:31 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Feb 13 16:01:31 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 16:01:31 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 16:01:31 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 16:01:31 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Feb 13 16:01:31 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 16:01:31 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 16:01:31 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 20:01:14 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 20:01:14 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 20:01:14 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Mon Feb 13 20:01:14 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 20:01:14 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 20:01:14 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Mon Feb 13 20:01:14 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 20:01:14 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Feb 13 20:01:14 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 20:01:14 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Mon Feb 13 20:01:14 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 20:01:14 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Mon Feb 13 20:01:14 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 08:19:45 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 08:19:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 08:19:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Feb 14 08:19:47 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 08:19:47 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 08:19:47 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Feb 14 08:19:47 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 08:19:47 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 08:19:48 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 08:19:48 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Feb 14 08:19:48 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 08:19:48 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 08:19:48 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 12:03:30 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 12:03:31 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 12:03:31 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Feb 14 12:03:31 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 12:03:31 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 12:03:31 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Feb 14 12:03:31 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 12:03:31 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 12:03:31 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 12:03:31 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Feb 14 12:03:31 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 12:03:31 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 12:03:31 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 16:01:43 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 16:01:43 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 16:01:43 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Feb 14 16:01:43 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 16:01:43 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 16:01:43 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Feb 14 16:01:43 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 16:01:44 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 16:01:44 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 16:01:44 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Feb 14 16:01:44 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 16:01:44 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 16:01:44 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 20:01:16 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 20:01:16 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 20:01:16 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Tue Feb 14 20:01:16 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 20:01:16 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 20:01:16 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Tue Feb 14 20:01:16 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 20:01:16 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Feb 14 20:01:16 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 20:01:16 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Tue Feb 14 20:01:16 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 20:01:16 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Tue Feb 14 20:01:16 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 08:28:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 08:28:57 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 08:28:58 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Feb 15 08:28:58 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 08:28:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 08:28:59 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Feb 15 08:28:59 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 08:28:59 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 08:28:59 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 08:28:59 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Feb 15 08:28:59 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 08:29:00 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 08:29:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 12:05:37 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 12:05:37 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 12:05:37 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Feb 15 12:05:37 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 12:05:37 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 12:05:37 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Feb 15 12:05:37 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 12:05:37 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 12:05:37 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 12:05:37 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Feb 15 12:05:37 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 12:05:37 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 12:05:37 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 16:01:38 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 16:01:38 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 16:01:38 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Feb 15 16:01:38 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 16:01:38 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 16:01:38 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Feb 15 16:01:38 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 16:01:40 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 16:01:40 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 16:01:40 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Feb 15 16:01:40 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 16:01:40 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 16:01:40 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 19:28:13 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 19:28:13 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 19:28:13 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Feb 15 19:28:13 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 19:28:13 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 19:28:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Feb 15 19:28:13 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 19:28:13 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 19:28:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 19:28:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Feb 15 19:28:13 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 19:28:13 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 19:28:13 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 20:13:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 20:13:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 20:13:59 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Wed Feb 15 20:14:00 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 20:14:00 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 20:14:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Wed Feb 15 20:14:00 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 20:14:00 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Feb 15 20:14:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 20:14:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Wed Feb 15 20:14:00 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 20:14:00 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Wed Feb 15 20:14:00 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 16 08:46:55 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 16 08:47:05 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 16 08:47:05 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Thu Feb 16 08:47:05 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 16 08:47:05 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 16 08:47:05 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Thu Feb 16 08:47:05 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 16 08:47:05 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 16 08:47:05 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 16 08:47:05 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Thu Feb 16 08:47:05 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 16 08:47:05 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Thu Feb 16 08:47:05 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis] From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 11:38:04 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Mar 31 16:33:48 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel @dbname=' returns '80' for the databases on the SQL Server. I am trying to avoid NULLs in my columns and am finding that the empty string works as a substitute for almost all cases where I would have used a NULL in the past. However, when I pass empty strings as values in the tuple second argument to the execute method (e.g. 'cu.execute(stmnt, tpl)'), the following exception is raised: ---------------------------------------------------------- (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', '', '', 'Identification of hemolytic granules isolated from human myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen Health University School of Medicine, Aichi, Japan.', 'eng', '', 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. 1990 May 15; 46 (5): 495-8') INSERT INTO pmCiteTest (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, PDMedDate, ArticleTitle, Pagination, Affiliation, Language, VernacularTitle, Country, MedlineTA, NlmUniqueID, PubDate, Cite) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Traceback (most recent call last): File "./cp2mstest.py", line 42, in ? cu_mx.execute(ins, tuple(r)) mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length or precision specification 0 is invalid.', 4579) ---------------------------------------------------------- When the exception is raised, I print the tuple and the SQL statement implicated in the exception before re-raising it. None of the columns in the pmCiteTest table allow NULLs. If I change instances of the empty string in the tuple to single spaces, the INSERTs work. However, I'd like to be able to insert empty strings passed in as parameter values. I don't want columns that should have an empty string to match "LIKE ' %'" conditions, and I don't want to have to generate the SQL statement dynamically. Is there a way I can pass in empty strings as parameter values to SQL Server through mx.ODBC? Thanks, Chuck Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From mal at egenix.com Mon Aug 9 19:16:17 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Mar 31 16:33:49 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC In-Reply-To: References: Message-ID: <4117A351.9080600@egenix.com> Charles Bearden wrote: > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > @dbname=' returns '80' for the databases on the SQL Server. > > I am trying to avoid NULLs in my columns and am finding that the empty > string works as a substitute for almost all cases where I would have > used a NULL in the past. However, when I pass empty strings as values > in the tuple second argument to the execute method (e.g. > 'cu.execute(stmnt, tpl)'), the following exception is raised: Have you tried the same thing on Windows using the MS SQL Server ODBC driver ? This is likely a problem with FreeTDS - mxODBC certainly doesn't have problems with empty string :-) > ---------------------------------------------------------- > (4, '2112097', '90269415', '0014-4754', '46', '5', '1990', 'May', '15', > '', '', 'Identification of hemolytic granules isolated from human > myocardial cells.', '495-8', 'Department of Microbiology, Fujita-gakuen > Health University School of Medicine, Aichi, Japan.', 'eng', '', > 'SWITZERLAND', 'Experientia', '0376547', '1990 May 15', 'Experientia. > 1990 May 15; 46 (5): 495-8') > > INSERT INTO pmCiteTest > (ID, PMID, MID, ISSN, Volume, Issue, PDYear, PDMonth, PDDay, PDSeason, > PDMedDate, ArticleTitle, Pagination, Affiliation, Language, > VernacularTitle, > > Country, MedlineTA, NlmUniqueID, PubDate, Cite) > VALUES > (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) > > Traceback (most recent call last): > File "./cp2mstest.py", line 42, in ? > cu_mx.execute(ins, tuple(r)) > mxODBC.OperationalError: ('', 1001, '[FreeTDS][SQL Server]Line 1: Length > or precision specification 0 is invalid.', 4579) > ---------------------------------------------------------- > > When the exception is raised, I print the tuple and the SQL statement > implicated in the exception before re-raising it. None of the columns > in the pmCiteTest table allow NULLs. > > If I change instances of the empty string in the tuple to single spaces, > the INSERTs work. However, I'd like to be able to insert empty strings > passed in as parameter values. I don't want columns that should have an > empty string to match "LIKE ' %'" conditions, and I don't want to have > to generate the SQL statement dynamically. Is there a way I can pass in > empty strings as parameter values to SQL Server through mx.ODBC? > > Thanks, > Chuck > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 09 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Mon Aug 9 12:18:30 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Mar 31 16:33:49 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Monday, August 09, 2004 11:16 AM > > Charles Bearden wrote: > > I'm using mx-commercial-2.0.6, iODBC 3.51.2, and FreeTDS freetds-0.62.4 > > on RH9 to talk to an MS SQL Server 2000, and 'exec sp_dbcmptlevel > > @dbname=' returns '80' for the databases on the SQL Server. > > > > I am trying to avoid NULLs in my columns and am finding that the empty > > string works as a substitute for almost all cases where I would have > > used a NULL in the past. However, when I pass empty strings as values > > in the tuple second argument to the execute method (e.g. > > 'cu.execute(stmnt, tpl)'), the following exception is raised: > > Have you tried the same thing on Windows using the MS SQL Server > ODBC driver ? Good question. I haven't, and I will. Thanks, Chuck > This is likely a problem with FreeTDS - mxODBC certainly doesn't > have problems with empty string :-) From info at egenix.com Tue Aug 10 11:39:38 2004 From: info at egenix.com (info@egenix.com) Date: Fri Mar 31 16:33:49 2006 Subject: [egenix-users] Re: [egenix-sales] Zope DA which closes connection automatically In-Reply-To: <000001c47e0e$82023080$6ac9280a@domw2k.int> References: <000001c47e0e$82023080$6ac9280a@domw2k.int> Message-ID: <20040810103938.1287.6@wonderland.1092123262.fake> On 2004-08-09 at 14:43:44 [+0200], Florian Reiser wrote: > Dear ladies and gentlemen, > > we use the ZODBC DA to connect to VFP dbf-tables. > When we have to do maintenance jobs on the database, > then we recognize, that Zope doesn't release any > database connection, no matter how long it wasn't used > anymore. > > This is very disturbing, when we try to clean up the database > and the associated tables. > > Is it possible to cause your Zope ODBC DA to release the connection > after some timeout or even better after every request? Dear Mr. Reiser, we assume you mean the mxODBCZopeDA and as we have nothing to do with the ZODBCDA. The mxODBDCZopeDA has several options that maybe of use to you. The option "connect on demand" will avoid the usual DA procedure of opening and keeping open the connection objects. If this does not work for you then you are still able to call .open() and close() methods directly on connection objects through ExternalMethods. Version 1.1 of the will include customisable timeouts for both connections and queries assuming that the ODBC driver itself supports this. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Tue Aug 10 13:25:37 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Mar 31 16:33:49 2006 Subject: [egenix-users] Inserting empty strings into SQL Server tables via mx.ODBC Message-ID: > > Have you tried the same thing on Windows using the MS SQL Server > > ODBC driver ? > > Good question. I haven't, and I will. > Thanks, Chuck > > > This is likely a problem with FreeTDS - mxODBC certainly doesn't > > have problems with empty string :-) FWIW, it does appear to be an issue with FreeTDS. I've had some useful responses on that list as well as a patch from Freddy Zigliano against the CVS, which I will build and try out. Thanks again, Chuck From Charles.F.Bearden at uth.tmc.edu Thu Aug 12 12:42:55 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Mar 31 16:33:49 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both talking to MS SQL Server 2000. When I try to insert strings longer than 255 characters into a MS SQL column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get the following exception: Traceback (most recent call last): File "./convtest.py", line 32, in ? try: mx_cur.execute(ins, t) mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed implicit conversion from data type text to data type nvarchar, table 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function to run this query.", 4579) Here's a sample script: ------------------------------------------------------------------------ -- #!/c/Python23/python import os os.environ['TDSDUMP'] = './tdsdumdumpdump' import mx.ODBC.Windows as mxodbc #import mx.ODBC.iODBC as mxodbc c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') cur = c.cursor() cur.execute('SET ARITHABORT ON') colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, pdseason, pdmeddate, articletitle, pagination, affiliation, language, vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?' t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', '', '', '', 'Recommended guidelines for uniform reporting of data from out-of-hospital cardiac arrest (new abridged version). The "Utstein style". The European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, and Australian Resuscitation Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') stmnt = ''' INSERT INTO pmCite (%s) VALUES (%s) ''' % (colnames, placeholders) cur.execute(stmnt, t) c.commit() c.close() ------------------------------------------------------------------------ -- This is the Windows version of the script, which doesn't raise the exception. On RH with Python 2.3.3, using iODBC, the exception is thrown. In other words, it appears that somewhere along the line, when the parameter list for the dynamic query is being built or something, the ArticleTitle column (beginning with 'Recommended guidelines') is being set up as TEXT rather than NVARCHAR (as the target table defines the column). I still don't understand what goes on behind the scenes of ODBC very well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, iODBC, or FreeTDS. Is there a way around having to run CONVERT on every column that might have nvarchar values larger than 255 bytes? Thanks for any pointers, Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From robbyd at veratest.com Fri Aug 13 15:20:47 2004 From: robbyd at veratest.com (Robby Dermody) Date: Fri Mar 31 16:33:49 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") Message-ID: <1092421246.12362.11.camel@luna> Hi guys, This problem has been causing me to pull my hair out for the last day or two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect to a MS SQL database via either unixODBC or iODBC. Relevant versions: mxODBC: 2.0.6 mx-base: 2.0.5 unixodbc: 2.2.4-9 iodbc: 3.51.2 freetds: 0.61 With both unixODBC and iODBC, I have installed them, configured odbc.ini, and can use their test programs to connect to this database and query it fine. No problems there. However, when I mix mxODBC into the picture and try to connect via that, I get something like the following: >>> db = mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") Traceback (most recent call last): File "", line 1, in ? mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', 5921) Here's the mxodbc.log output (when compiled in 'debug' mode): mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=1 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', clearAC=0 mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. I get the same output from iodbc, "Option not supported". Any idea how to find out WHAT option they're talking about? Note I get the same error when using Connect(), and as you can see, I've tried it with both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each time. I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and that it has nothing to do with freeTDS or the MS SQL server itself. Any help would be welcome!! Thanks, Robby From mal at egenix.com Tue Aug 17 10:43:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Mar 31 16:33:49 2006 Subject: [egenix-users] MS SQL ODBC connect problem ("Option not supported") In-Reply-To: <1092421246.12362.11.camel@luna> References: <1092421246.12362.11.camel@luna> Message-ID: <4121B728.4030708@egenix.com> Robby Dermody wrote: > Hi guys, > > This problem has been causing me to pull my hair out for the last day or > two. I'm trying to use mxODBC under Debian Linux 'unstable' to connect > to a MS SQL database via either unixODBC or iODBC. > > Relevant versions: > mxODBC: 2.0.6 > mx-base: 2.0.5 > unixodbc: 2.2.4-9 > iodbc: 3.51.2 > freetds: 0.61 > > With both unixODBC and iODBC, I have installed them, configured > odbc.ini, and can use their test programs to connect to this database > and query it fine. No problems there. However, when I mix mxODBC into > the picture and try to connect via that, I get something like the > following: > > >>>>db = > > mx.ODBC.unixODBC.DriverConnect("DSN=testdsn;UID=testuser;PWD=pass;") > Traceback (most recent call last): > File "", line 1, in ? > mxODBC.NotSupportedError: ('S1C00', 1, '[unixODBC]Option not supported', > 5921) > > > Here's the mxodbc.log output (when compiled in 'debug' mode): > > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=1 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 mxODBC_New_UseDriverConnect: failed. > mxODBC_New_UseDriverConnect: dsn='DSN=testdsn;UID=testuser;PWD=pass;', > clearAC=0 > mxODBC_ReportError: sqlstate='S1C00' code=1 text='[unixODBC]Option not > supported' lineno=5921 rc=-1 > mxODBC_New_UseDriverConnect: failed. > > > I get the same output from iodbc, "Option not supported". Any idea how > to find out WHAT option they're talking about? Note I get the same error > when using Connect(), and as you can see, I've tried it with > both clear_auto_commit=1 and clear_auto_commit=0 ...same thing each > time. > > I'm thinking this is a problem between mxODBC and unixodbc/iodbc....and > that it has nothing to do with freeTDS or the MS SQL server itself. mxODBC does work with FreeTDS, so the problem must be with your setup. Two things: 1. try to remove the final ";" in your DSN 2. check your odbc.ini file whether you are using unsupported connection options. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Wed Aug 18 15:12:48 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Mar 31 16:33:49 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT In-Reply-To: References: Message-ID: <412347C0.204@egenix.com> Charles Bearden wrote: > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > talking to MS SQL Server 2000. > > When I try to insert strings longer than 255 characters into a MS SQL > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > the following exception: > > Traceback (most recent call last): > File "./convtest.py", line 32, in ? > try: mx_cur.execute(ins, t) > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > implicit conversion from data type text to data type nvarchar, table > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > to run this query.", 4579) The problem here is that FreeTDS tries to convert the Python 8-bit string input into a UTF-16 string. Obviously SQL Server doesn't want to do this conversion implicitly (probably because it doesn't know the encoding of the input string). Have you tried passing in the data as Unicode string ? > Here's a sample script: > ------------------------------------------------------------------------ > -- > #!/c/Python23/python > > import os > os.environ['TDSDUMP'] = './tdsdumdumpdump' > import mx.ODBC.Windows as mxodbc > #import mx.ODBC.iODBC as mxodbc > c = mxodbc.DriverConnect('DSN=dbname;UID=aUser;PWD=password') > cur = c.cursor() > cur.execute('SET ARITHABORT ON') > > colnames = 'id, pmid, mid, issn, volume, issue, pdyear, pdmonth, pdday, > pdseason, pdmeddate, articletitle, pagination, affiliation, language, > vernaculartitle, country, medlineta, nlmuniqueid, pubdate, cite' > > placeholders = '?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, > ?, ?' > > t = (5300, '1389710', '93001812', '0007-0769', '67', '4', '1992', 'Apr', > '', '', '', 'Recommended guidelines for uniform reporting of data from > out-of-hospital cardiac arrest (new abridged version). The "Utstein > style". The European Resuscitation Council, American Heart Association, > Heart and Stroke Foundation of Canada, and Australian Resuscitation > Council.', '325-33', '', 'eng', '', 'ENGLAND', 'Br Heart J', '0370634', > '1992 Apr', 'Br Heart J. 1992 Apr; 67 (4): 325-33') > > stmnt = ''' > INSERT INTO pmCite > (%s) > VALUES > (%s) > ''' % (colnames, placeholders) > cur.execute(stmnt, t) > c.commit() > c.close() > ------------------------------------------------------------------------ > -- > > This is the Windows version of the script, which doesn't raise the > exception. On RH with Python 2.3.3, using iODBC, the exception is > thrown. > > In other words, it appears that somewhere along the line, when the > parameter list for the dynamic query is being built or something, the > ArticleTitle column (beginning with 'Recommended guidelines') is being > set up as TEXT rather than NVARCHAR (as the target table defines the > column). > > I still don't understand what goes on behind the scenes of ODBC very > well, so I don't know if this is occurring in mx.ODBC, mx.ODBC.iODBC, > iODBC, or FreeTDS. > > Is there a way around having to run CONVERT on every column that might > have nvarchar values larger than 255 bytes? > > Thanks for any pointers, > > Chuck Bearden > Systems Analyst III > School of Health Information Sciences > University of Texas at Houston > 713.500.3954 (voice) > 713.500.3907 (fax) > Charles.F.Bearden@uth.tmc.edu > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 18 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 16:37:52 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Mar 31 16:33:49 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: Wednesday, August 18, 2004 7:13 AM > > Charles Bearden wrote: > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on RH 9, > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, both > > talking to MS SQL Server 2000. > > > > When I try to insert strings longer than 255 characters into a MS SQL > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I get > > the following exception: > > > > Traceback (most recent call last): > > File "./convtest.py", line 32, in ? > > try: mx_cur.execute(ins, t) > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > implicit conversion from data type text to data type nvarchar, table > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT function > > to run this query.", 4579) > > The problem here is that FreeTDS tries to convert the Python > 8-bit string input into a UTF-16 string. Obviously SQL Server > doesn't want to do this conversion implicitly (probably because > it doesn't know the encoding of the input string). > > Have you tried passing in the data as Unicode string ? Thanks! That's extremely helpful to know. I don't think I have tried passing in a Unicode string, or if I have there was some other place in the program where it caused a UnicodeDecodeError or something. I may retool the script to implement your suggestion. The solution I came up with is a bit more kludgy. I wrote a function that takes the data from the '.description' attribute of the source database cursor, after executing a 'select * from limit 1' to populate it correctly, in order to build an INSERT statmemt with 'CONVERT(?)' instead of simply '?' for those columns that might have data longer than 255 bytes: ------------------------------------------------------ def mkIns(tname, cur): cur.execute('select * from %s limit 1' % (tname,)) colinfo_lst = map(lambda t: (t[0],t[1],t[3]), cur.description) clist = [] ; phlist = [] ## column name list and placeholder list for colinfo in colinfo_lst: colname, coltype, colsize = colinfo clist.append(colname) #-- 1043 is the PostgreSQL object id for type VARCHAR if coltype == 1043 and colsize > 250: phlist.append('CONVERT(nvarchar, ?)') else: phlist.append('?') #-- put it all together and return return ''' INSERT INTO %s (%s) VALUES (%s) ''' % (tname, ', '.join(clist), ', '.join(phlist)) ------------------------------------------------------ It's an ugly hack, but it seems to do the trick. Thanks again for your response. Chuck From joseph.kocherhans at hsc.utah.edu Wed Aug 18 16:22:57 2004 From: joseph.kocherhans at hsc.utah.edu (Joseph Kocherhans) Date: Fri Mar 31 16:33:49 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 not found Message-ID: I've been having trouble with the packaged versions of iodbc and freetds on debian, so I decided to try building them both from source. I put iodbc in /opt/iodbc, and tested the freetds odbc driver with iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: Extension('mx.ODBC.iODBC.mxODBC', ['mx/ODBC/iODBC/mxODBC.c', 'mx/ODBC/iODBC/mxSQLCodes.c' ], include_dirs=['mx/ODBC/iODBC', '/opt/iodbc/include'], define_macros=[('iODBC', None), ('MAX_COLUMN_NAME_LENGTH', '50')], library_dirs=['/opt/iodbc/lib'], libraries=['iodbc'] ), Everything builds fine, (running "python2.3 setup.py install") but when I try to import mx.ODBC.iODBC in python2.3, I get the following error: Traceback (most recent call last): File "/home/jkocherhans/mxodbctest.py", line 1, in ? from mx.ODBC.iODBC import DriverConnect File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", line 8, in ? from mxODBC import * ImportError: libiodbc.so.2: cannot open shared object file: No such file or directory Is there something besides include_dirs and library_dirs I need to change to use the iODBC I installed in /opt? Thanks, Joseph Kocherhans From Charles.F.Bearden at uth.tmc.edu Wed Aug 18 17:32:11 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Mar 31 16:33:49 2006 Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 notfound Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Joseph Kocherhans > Sent: Wednesday, August 18, 2004 4:23 PM > To: egenix-users@lists.egenix.com > Subject: [egenix-users] Non-standard location for iODBC, libiodbc.so.2 > notfound > > I've been having trouble with the packaged versions of iodbc and > freetds on debian, so I decided to try building them both from source. > I put iodbc in /opt/iodbc, and tested the freetds odbc driver with > iodbctest. Everything looks good. I modified mxCOMMERCIAL.py to: > > Extension('mx.ODBC.iODBC.mxODBC', > ['mx/ODBC/iODBC/mxODBC.c', > 'mx/ODBC/iODBC/mxSQLCodes.c' > ], > include_dirs=['mx/ODBC/iODBC', > '/opt/iodbc/include'], > define_macros=[('iODBC', None), > ('MAX_COLUMN_NAME_LENGTH', '50')], > library_dirs=['/opt/iodbc/lib'], > libraries=['iodbc'] > ), > > Everything builds fine, (running "python2.3 setup.py install") but when > I try to import mx.ODBC.iODBC in python2.3, I get the following error: > > Traceback (most recent call last): > File "/home/jkocherhans/mxodbctest.py", line 1, in ? > from mx.ODBC.iODBC import DriverConnect > File "/usr/lib/python2.3/site-packages/mx/ODBC/iODBC/__init__.py", > line 8, in ? > from mxODBC import * > ImportError: libiodbc.so.2: cannot open shared object file: No such > file or directory > > Is there something besides include_dirs and library_dirs I need to > change to use the iODBC I installed in /opt? Hi Joseph, Even though you tell mxCOMMERCIAL.py where to find the iODBC libraries when building, the system won't know where they are when running applications. You need to do one of two things: 1) set the LD_LIBRARY_PATH environment variable to include '/opt/iodbc/lib' in each session that expects to use mx.ODBC.iODBC; or 2) add '/opt/iodbc/lib' to the list of library paths in /etc/ld.so.conf, and than run ldconfig to update the system-wide cache of library links. You can run it with the '-v' flag and pipe through less to see the directories it processes. The second solution is usually better because it is more transparent to users, but you do need to be root to update the system-wide /etc/ld.so.cache. The first solution is simple to implement via the Python os.environ dictionary of environment variables, but it must be done in every script that needs to load shared libraries from unorthodox locations. If you are installing to /opt, I'll bet you have permissions to run ldconfig. Hope that helps out. Chuck Bearden Systems Analyst III School of Health Information Sciences University of Texas at Houston 713.500.3954 (voice) 713.500.3907 (fax) Charles.F.Bearden@uth.tmc.edu From Charles.F.Bearden at uth.tmc.edu Thu Aug 19 12:18:21 2004 From: Charles.F.Bearden at uth.tmc.edu (Charles Bearden) Date: Fri Mar 31 16:33:49 2006 Subject: [egenix-users] Max length of (N)VARCHAR; min length of TEXT Message-ID: > -----Original Message----- > From: egenix-users-bounces@lists.egenix.com [mailto:egenix-users- > bounces@lists.egenix.com] On Behalf Of Charles Bearden > Sent: Wednesday, August 18, 2004 3:38 PM > > > -----Original Message----- > > From: M.-A. Lemburg [mailto:mal@egenix.com] > > Sent: Wednesday, August 18, 2004 7:13 AM > > > > Charles Bearden wrote: > > > Egenix-mx-commercial-2.0.6, libiodbc-3.51.2, and freetds-0.62.4 on > RH 9, > > > and egenix-mx-commercial-2.0.6 with Windows ODBC driver on WinXP, > both > > > talking to MS SQL Server 2000. > > > > > > When I try to insert strings longer than 255 characters into a MS > SQL > > > column defined as nvarchar(1024) through FreeTDS/iODBC from Linux, I > get > > > the following exception: > > > > > > Traceback (most recent call last): > > > File "./convtest.py", line 32, in ? > > > try: mx_cur.execute(ins, t) > > > mxODBC.OperationalError: ('', 260, "[FreeTDS][SQL Server]Disallowed > > > implicit conversion from data type text to data type nvarchar, table > > > 'IfSH_Test.dbo.pmCite', column 'ArticleTitle'. Use the CONVERT > function > > > to run this query.", 4579) > > > > The problem here is that FreeTDS tries to convert the Python > > 8-bit string input into a UTF-16 string. Obviously SQL Server > > doesn't want to do this conversion implicitly (probably because > > it doesn't know the encoding of the input string). > > > > Have you tried passing in the data as Unicode string ? > > Thanks! That's extremely helpful to know. I don't think I have tried > passing in a Unicode string, or if I have there was some other place in > the program where it caused a UnicodeDecodeError or something. I may > retool the script to implement your suggestion. When I try to use Unicode strings as arguments to an SQL statement using placeholders and represented as a (non-Unicode) byte-string, I get a UnicodeEncodeError like "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 128: ordinal not in range(128)". I believe that this occurs because some part of the back end (mx.ODBC? FreeTDS? iODBC?) is trying to encode the Unicode arguments in the same encoding as the SQL statement appears to be in. When I then try to use an SQL statement that is a Python Unicode string with arguments that are Unicode strings, I get a TypeError: "TypeError: SQL command must be a string". So, the setup doesn't appear to like Unicode values with byte-string SQL statements, and it appears not to like Unicode SQL statements. At present, I don't see a way to pass the data in as Python Unicode strings. Am I missing something? Thanks, Chuck > The solution I came up with is a bit more kludgy. I wrote a function > that takes the data from the '.description' attribute of the source > database cursor, after executing a 'select * from limit 1' > to populate it correctly, in order to build an INSERT statmemt with > 'CONVERT(?)' instead of simply '?' for those columns that might have > data longer than 255 bytes: [ugly hack ellipsis]