[egenix-users] Data truncation by Microsoft ODBC driver for NVARCHAR

[M.-A. Lemburg]

Hi Jan,

just to clarify: you have the field filled with 29 characters
and the last one is a character which needs two bytes UTF-8
representation ?

You may want to try to set the connection's .stringformat
to NATIVE_UNICODE_STRINGFORMAT. This will result in mxODBC
requesting data as Unicode.

However, please note that your specific case may also be
a bug in the driver, since these often use UTF-8 strings
internally to store Unicode data and then "forget" to
adjust the buffer lengths to accommodate for the increase
in size when the strings have multi-byte representations.

If you could provide an example and specific driver and database
versions, we can try to replicate the problem.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Sep 26 2017)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...           http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
                      http://www.malemburg.com/


On 26.09.2017 15:08, Jan Murre wrote:
> Hi,
> 
> I am query-ing a MS SQL database from Redhat Linux using the "Microsoft
> ODBC Driver 13 for SQL Server".
> 
> There is a NVARCHAR(30) field in our database that is filled with data
> having a 2-byte utf-8 char on the last position. When query-ing, the ODBC
> driver issues this warning:
> 
> mx.ODBC.Error.Warning: ('01004', 0, '[Microsoft][ODBC Driver 13 for SQL
> Server]String data, right truncation', 8668)
> 
> This results in corrupted data in the resultsset, because only the first
> byte of this 2-byte utf-8 char is in the column.
> 
> I tried with serveral settings for 'connection.encoding' and
> 'connections.stringformat', but without success.
> 
> Is this an ODBC driver issue? Would it be possible to work around this with
> certain settings of mxODBC?
> 
> Regards, Jan
> 
> 
> 
> 
> _______________________________________________________________________
> eGenix.com User Mailing List                     http://www.egenix.com/
> https://www.egenix.com/mailman/listinfo/egenix-users
>