[egenix-users] Using mxodbc with SQL Server 2012
Sheila Allen
sallen at zeomega.com
Thu Sep 3 02:44:58 CEST 2015
> I'm looking into the MS SQL Server Native client again as well, we tried
it when they first announced it and it would segfault with any significant
load
We're currently testing the MS ODBC Driver for Linux, and ran into a
similar segfault under load. This was resolved by upgrading unixODBC 2.3.0
to 2.3.2, even though the MS ODBC Driver officially only supports 2.3.0.
Some discussion of the issue here:
https://social.msdn.microsoft.com/Forums/sqlserver/en-US/23fafa84-d333-45ac-8bd0-4b76151e8bcc/sql-server-driver-for-linux-causes-segmentation-fault
I wish Microsoft would show some signs that they plan to maintain the MS
ODBC Driver for Linux, such as acknowledging this issue and providing an
install script that supports the newer unixODBC 2.3.2.
Here is an article on making it work on Debian, along with some useful
comments at the bottom.
https://blog.afoolishmanifesto.com/posts/install-and-configure-the-ms-odbc-driver-on-debian/
On Wed, Sep 2, 2015 at 3:55 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> On 02.09.2015 22:14, John Anderson wrote:
> > On Wed, Sep 2, 2015 at 1:05 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> >
> >> On 02.09.2015 20:36, John Anderson wrote:
> >>> Hey, We currently use pymssql and it was recommended that we checkout
> >>> mxODBC again (its been a few years) and even though most of our
> projects
> >>> require python 3 we have a few legacy python applications that are
> still
> >> on
> >>> python 2.7 so I thought I would give it a try.
> >>>
> >>> I'm currently getting this error:
> >>>
> >>> mx.ODBC.Error.OperationalError: ('08003', 0, '[unixODBC][Driver
> >>> Manager]Connnection does not exist', 13710)
> >>
> >> This is a typical error message you get when the connection is found
> >> by the ODBC manager, i.e. it is configured in the odbc.ini file,
> >> but it cannot load it.
> >>
> >> This can be a permission problem or a linker problem. You can
> >> try this to check:
> >>
> >> ldd /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
> >>
> >> Also see our documentation for FreeTDS:
> >>
> >> http://www.egenix.com/products/python/mxODBC/doc/#_Toc427692966
> >>
> >> A note on FreeTDS:
> >>
> >> You should really use the MS SQL Server Native Client for
> >> Linux, if you can. It is much more feature complete compared
> >> to FreeTDS and also much easier to configure:
> >>
> >> https://msdn.microsoft.com/en-us/library/hh568451%28v=sql.110%29.aspx
> >>
> >> These are the instructions from our documentation:
> >>
> >> http://www.egenix.com/products/python/mxODBC/doc/#_Toc427692959
> >>
> >>
> > I figured it is some type of config problem but its really odd because
> > SQLAlchemy can make it work just fine, this lets me connect and query:
> >
> > from sqlalchemy import create_engine
> > engine = create_engine('mssql+mxodbc://<secret>:<secret>@mt1')
>
> SA translates the connect to a call like this:
>
> db = mx.Connect('mt1', '<UID>', '<PWD>')
>
> It's possible that naming both the DSN and the TDS server using
> the same identifer confuses some part in the setup when using
> DriverConnect(). The latter is more flexible, though, since it
> allows adding more connection options.
>
> An ODBC trace will usually show more information about what exactly
> is not working. The mxODBC documentation explains how this
> can be enabled (but I see below that you have that enabled already).
>
> > I'm looking into the MS SQL Server Native client again as well, we tried
> it
> > when they first announced it and it would segfault with any significant
> > load and they mostly support Red Hat instead of Ubuntu. I want to
> compare
> > all variations of it.
> >
> > The reason we are finally looking at other drivers is because the pymssql
> > driver isn't quite fast enough, when selecting 2 million records it takes
> > about 10 seconds:
> >
> > $ time python pymssql_query.py
> > 2148068
> >
> > real 0m11.138s
> > user 0m9.606s
> > sys 0m0.724s
> >
> > With mxODBC + SQLAlchemy (since I can't get mxodbc to work without SA):
> >
> > $ time python mxodbc_sa.py
> > 2148068
> >
> > real 0m49.407s
> > user 0m19.013s
> > sys 0m27.161s
> >
> > So it looks like mxodbc is significantly slower than pymssql even though
> > both are utilizing FreeTDS behind the scenes.
>
> The SA code is likely doing a lot more on top of just running
> the query. You also have tracing enabled, which slows down things
> a lot.
>
> There are also several options to optimize queries to SQL Server
> using mxODBC (e.g. using read-only connections, different
> transaction isolations, direct execution, NO COUNT).
>
> We also have the mxODBC Connect product which removes the need for
> a Unix ODBC driver altogether and uses the MS driver on the
> Windows database server to directly communicate via shared
> memory with the database kernel. mxODBC Connect takes care of
> the client-server communication and is optimized for the
> Python DB-API way of fetching data.
>
> > I'm going to try the native driver next and then I'll let you know.
> >
> > Here is the output of the ldd command:
> >
> > ldd /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
> > linux-vdso.so.1 => (0x00007ffec93f6000)
> > libodbcinst.so.2 => /usr/lib/x86_64-linux-gnu/libodbcinst.so.2
> > (0x00007ff192bcd000)
> > libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2
> > (0x00007ff192985000)
> > libgnutls-deb0.so.28 => /usr/lib/x86_64-linux-gnu/libgnutls-deb0.so.28
> > (0x00007ff19266a000)
> > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
> > (0x00007ff19244c000)
> > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff192082000)
> > libltdl.so.7 => /usr/lib/x86_64-linux-gnu/libltdl.so.7
> (0x00007ff191e77000)
> > libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3
> (0x00007ff191ba8000)
> > libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3
> > (0x00007ff191977000)
> > libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2
> > (0x00007ff191772000)
> > libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0
> > (0x00007ff191567000)
> > libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007ff19134c000)
> > libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0
> > (0x00007ff191105000)
> > libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6
> > (0x00007ff190ef1000)
> > libnettle.so.4 => /usr/lib/x86_64-linux-gnu/libnettle.so.4
> > (0x00007ff190cc0000)
> > libhogweed.so.2 => /usr/lib/x86_64-linux-gnu/libhogweed.so.2
> > (0x00007ff190a91000)
> > libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10
> (0x00007ff190811000)
> > /lib64/ld-linux-x86-64.so.2 (0x00007ff193066000)
> > libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ff19060d000)
> > libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1
> > (0x00007ff190408000)
> > libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2
> (0x00007ff1901ec000)
> > libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007ff18ffe3000)
>
> Looks fine, so it's probably not related to the linker.
>
> >>> with this code:
> >>>
> >>> import mx.ODBC.unixODBC as mx
> >>> db = mx.DriverConnect('DSN=mt1;UID=<secret>;PWD=<secret>')
> >>> c = db.cursor()
> >>> c.tables()
> >>> res = c.fetchall()
> >>>
> >>> and I'm wondering might cause that because I can use isql/tsql and it
> >> works:
> >>>
> >>> tsql -S mt1 -U <secret> -P <secret>
> >>> isql mt1 <secret> <secret>
> >>>
> >>> osql is the only one that doesn't work:
> >>>
> >>>
> >>> osql -S mt1 -U <secret> -P <secret>
> >>>
> >>> checking shared odbc libraries linked to isql for default
> directories...
> >>> strings: '': No such file
> >>> trying /tmp/sql ... no
> >>> trying /tmp/sql ... no
> >>> trying /etc ... OK
> >>> checking odbc.ini files
> >>> reading /home/sontek/.odbc.ini
> >>> [mt1] not found in /home/sontek/.odbc.ini
> >>> reading /etc/odbc.ini
> >>> [mt1] found in /etc/odbc.ini
> >>> found this section:
> >>> [mt1]
> >>> Driver = FreeTDS
> >>> Description = ODBC connection via FreeTDS
> >>> Trace = Yes
> >>> TraceFile = /tmp/sql.log
> >>> ServerName = mt1
> >>> Database = SM_AccountsNew
> >>> looking for driver for DSN [mt1] in /etc/odbc.ini
> >>> found driver line: " Driver = FreeTDS"
> >>> driver "FreeTDS" found for [mt1] in odbc.ini
> >>> found driver named "FreeTDS"
> >>> "FreeTDS" is not an executable file
> >>> looking for entry named [FreeTDS] in /etc/odbcinst.ini
> >>> found driver line: " Driver =
> >>> /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so"
> >>> found driver /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so for
> [FreeTDS]
> >>> in odbcinst.ini
> >>> /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so is not an executable file
> >>> osql: error: no driver found for mt1
> >>>
> >>>
> >>> Anyone know what I might be missing?
> >>>
> >>> Here is my FreeTDS:
> >>>
> >>> [mt1]
> >>> host = mt1-db01.corp.surveymonkey.com
> >>> port = 1433
> >>> tds version = 7.3
> >>> client charset = UTF-8
> >>>
> >>> odbc.ini:
> >>>
> >>> [mt1]
> >>> Driver = FreeTDS
> >>> Description = ODBC connection via FreeTDS
> >>> Trace = Yes
> >>> TraceFile = /tmp/sql.log
> >>> ServerName = mt1
> >>> Database = SM_AccountsNew
> >>>
> >>>
> >>> odbcinst.ini:
> >>> [FreeTDS]
> >>> Description = TDS
> >>> Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
> >>> Startup = /usr/lib/x86_64-linux-gnu/odbc/libtdsS.so
> >>> UsageCount = 1
> >>> Threading = 2
> >>>
> >>>
> >>>
> >>>
> >>> _______________________________________________________________________
> >>> eGenix.com User Mailing List
> http://www.egenix.com/
> >>> https://www.egenix.com/mailman/listinfo/egenix-users
> >>>
> >>
> >> --
> >> Marc-Andre Lemburg
> >> eGenix.com
> >>
> >> Professional Python Services directly from the Source (#1, Sep 02 2015)
> >>>>> Python Projects, Coaching and Consulting ... http://www.egenix.com/
> >>>>> mxODBC Plone/Zope Database Adapter ... http://zope.egenix.com/
> >>>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
> >> ________________________________________________________________________
> >> 2015-08-27: Released eGenix mx Base 3.2.9 ...
> http://egenix.com/go83
> >>
> >> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
> >>
> >> eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
> >> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
> >> Registered at Amtsgericht Duesseldorf: HRB 46611
> >> http://www.egenix.com/company/contact/
> >>
> >
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source (#1, Sep 02 2015)
> >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/
> >>> mxODBC Plone/Zope Database Adapter ... http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
> ________________________________________________________________________
> 2015-08-27: Released eGenix mx Base 3.2.9 ... http://egenix.com/go83
>
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
>
> eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
> Registered at Amtsgericht Duesseldorf: HRB 46611
> http://www.egenix.com/company/contact/
>
>
> _______________________________________________________________________
> eGenix.com User Mailing List http://www.egenix.com/
> https://www.egenix.com/mailman/listinfo/egenix-users
>
--
Sheila Allen | Software Architect
ZeOmega | Population Health Management Solutions
Improving Population Health One Person at a Time
Office: +1 214 618 9880 ext 8006 | Mobile: +1 214 491 7146
6200 Tennyson Parkway | Suite 200 | Plano, Texas 75024 USA
www.ZeOmega.com <http://www.zeomega.com/>
--
This e-mail message (including any attachments) may contain information
that is confidential, protected by the attorney-client or other applicable
privileges, or otherwise comprising non-public information. This message is
intended to be conveyed only to the designated recipient(s). If you have
any reason to believe you are not an intended recipient of this message,
please notify the sender by replying to this message and then deleting it
from your system. Any use, dissemination, distribution, or reproduction of
this message by unintended recipients is not authorized and may be unlawful.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: /mailman-archives/egenix-users/attachments/20150903/0d4bccfc/attachment.htm
More information about the egenix-users
mailing list