pyodbc: ODBC data type -155 is not supported.
Hello,
I’m working with a MSSQL database that uses DATETIMEOFFSET’s for date storage. However the type is unsupported, would you consider adding it in? DATETIMEOFFSET is an “UPPERCASE” type and as I understand quite a common way to store TZ info with a timestamp in SQL Server.
I can work around it with casting, but it would be great to see proper support. Finally, thanks for all your work on this so far as well, really happy with it and it is serving me very well 😃
Example: 2016-08-17 10:08:56.18 +00:00
I’m using v3.0.10 on Windows with Python 3.5.2 if that helps.
About this issue
- Original URL
- State: open
- Created 8 years ago
- Reactions: 10
- Comments: 24 (4 by maintainers)
Links to this issue
Commits related to this issue
- Base mssql.DATETIMEOFFSET on DateTime Fixed the base class of the :class:`.mssql.DATETIMEOFFSET` datatype to be based on the :class:`.DateTime` class hierarchy, as this is a datetime-holding datatype... — committed to sqlalchemy/sqlalchemy by zzzeek 5 years ago
- Base mssql.DATETIMEOFFSET on DateTime Fixed the base class of the :class:`.mssql.DATETIMEOFFSET` datatype to be based on the :class:`.DateTime` class hierarchy, as this is a datetime-holding datatype... — committed to sumau/sqlalchemy by zzzeek 5 years ago
If it helps, I just wrote an output converter function for this.
The binary format is this:
struct.Struct("<6hI2h")…which unpacks 20 bytes of data into a tuple of 9 values:year, month, day, hours, minutes, seconds, nanoseconds, offset_hours, offset_minutes…which can be used to construct a datetime object.If the timezone offset is negative, both offset_hours and offset_minutes will be negative.
You’ll need a tzinfo implementation if you want your datetimes to be timezone aware - FixedOffsetTimezone from psycopg2.tz does the job. Also remember that datetime’s constructor accepts microseconds, not nanoseconds:
total_offset_minutes = offset_hours * 60 + offset_minutesresult = datetime.datetime(year, month, day, hours, minutes, seconds, nanoseconds // 1000, tzinfo=FixedOffsetTimezone(total_offset_minutes))My understanding is that exactly what pyodbc supports will depend on exactly what constants are defined in headers like
sql.h,sqlext.h, ormsodbcsql.hwhen it’s built and installed. That could explain why it works on one system and not another.In case this is useful to anyone else, here is a short write-up of an investigation into this that I did a while ago while trying to fix this in Django. (I ended up with a janky monkeypatch that I’m not going to recommend using.) I’m not entirely sure all of this is correct at this point, either, and I’d be happily corrected on any of it.
SQL type -155 is
SQL_SS_TIMESTAMPOFFSET, which you can find documented:https://docs.microsoft.com/en-us/sql/relational-databases/native-client-odbc-date-time/data-type-support-for-odbc-date-and-time-improvements?view=sql-server-2017
Since this is a SQL-Server-specific extension to ODBC, it’s defined only in
msodbcsql.h(e.g./opt/microsoft/msodbcsql17/include/msodbcsql.h), and not in/usr/include/sql.horsqlext.has provided byunixodbc-dev.If you grab the pyodbc source:
https://files.pythonhosted.org/packages/a0/14/d23278740dd64aa47eea396c4caf5a1b464dacaf339dca21b9afbc973e4b/pyodbc-4.0.22.tar.gz
…you can see in
src/pydobc.hthat it’s getting its values for those constants likeSQL_SS_TIMESTAMPOFFSETfromsql.handsqlext.h, and it uses those (e.g. insrc/getdata.cpp) to figure out how to process data.Those in turn get baked into
site-packages/sql_server/pyodbc/introspection.pywhen you install the package.So to fix this “correctly”, we would need to patch pyodbc to make it include Microsoft’s custom extensions. This seems like an unlikely patch to make it upstream.
So why does this work with older versions of Django/Python/pyodbc? I’m not sure, but I think there might have previously been some kind of automatic or catch-all conversion to another datetime type. I haven’t had a chance to go through the commit history and look for that.
SQLAlchemy here .
How we doing on this? I’m getting hit on this now. I’m using the Microsoft drivers on linux.
@vdurante - I’ve been working on an enhancement to SQLAlchemy to add
datetimeoffsetsupport to themssql+pyodbcdialect. I’m hoping that it will be included in the next release. See this comment for (IMO) the underlying reason why support fordatetimeoffsetis not baked into pyodbc itself.Further to comment from @v-chojas:
Situation described by @MichaelE919 is almost certainly that the Windows machine was using the ancient “SQL Server” driver (shipped with Windows) while the Linux machine was using a newer “ODBC Driver xx for SQL Server”. The former returns the value as a string, while the latter returns a type -155 value which requires an output converter function to interpret the results. The issue is also documented in the wiki here.
Which ODBC driver is being used is also important, because the server will not advertise the newer types to those older drivers which do not support them:
https://docs.microsoft.com/en-us/sql/relational-databases/native-client-odbc-date-time/enhanced-date-and-time-type-behavior-with-previous-sql-server-versions-odbc