pyodbc: UTF-16-LE decode error

Environment

To diagnose, we usually need to know the following, including version numbers. On Windows, be sure to specify 32-bit Python or 64-bit:

Python: 3.7.4
pyodbc: 4.0.27
OS: Docker image = python:3.7-slim-stretch
DB: SQL Server (on Azure)
driver: ODBC Driver 17 for SQL Server

Running flask under gunicorn gevent worker

Issue

Randomly I receive the following error:

File "/usr/local/lib/python3.7/encodings/utf_16_le.py", line 15, in decode
def decode(input, errors='strict'):
File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base.py", line 196, in handle_abort
     sys.exit(1)
SystemExit: 1
The above exception was the direct cause of the following exception:
...
   File "/code/src/services/mssql.py", line 87, in execute_query
     cur.execute(query)

Tried reversing to 4.0.23 and still no good!

About this issue

Original URL
State: open
Created 5 years ago
Reactions: 3
Comments: 19 (5 by maintainers)

Most upvoted comments

Also he mentions it’s random. The system error which is thrown occurs randomly as the characters that are pulled (when they shouldn’t be) and ran through unicode_decode are random.

So, randomly unicode_decode gets a character it can’t decode and throws the system error instead of a pyodbc.error.

If I can find a Docker instance for Oracle, I can setup a reproducible test case for you, but I don’t use Docker regularly, so it may be a bit before I can get this done.

reedjosh on May 7, 2020

Connection class

import os

import pyodbc
from opencensus.trace import execution_context, status

from helpers import Formatter, Singleton
from helpers.config import MSSQL
from logger import new

LOG = new(__name__)


class MsSQLService(metaclass=Singleton):
    def __init__(self):
        try:
            tracer = execution_context.get_opencensus_tracer()
        except Exception:
            LOG.error("Failed to trace request", exc_info=True)
            raise

        try:
            with tracer.span(name=MSSQL["server_name"]) as _:
                tracer.add_attribute_to_current_span("dependency.type", "SQL")
                self.conn = pyodbc.connect(
                    (
                        "Driver={{ODBC Driver 17 for SQL Server}};Server={};"
                        "Database={};UID={};PWD={};"
                    ).format(
                        MSSQL["server_name"],
                        MSSQL["database"],
                        MSSQL["username"],
                        MSSQL["password"],
                    ),
                    autocommit=True,
                )
        except Exception as e:
            LOG.error(f"Connection exception: {e}")
            raise

    def execute_query(self, query, formatted=True):
        try:
            tracer = execution_context.get_opencensus_tracer()
        except Exception:
            LOG.error("Failed to get tracer", exc_info=True)
            raise

        with tracer.span(name=MSSQL["server_name"]) as _:
            tracer.add_attribute_to_current_span("dependency.type", "SQL")
            tracer.add_attribute_to_current_span("database.query", query)
            try:
                cur = self.conn.cursor()
                cur.execute(query)
                if formatted is True:
                    data = Formatter.format_cursor_mssql(cur)
                else:
                    data = None
            except Exception as e:
                LOG.info(f"MSSQL error: {e}")
                LOG.info(f"MSSQL error cause: {e.__cause__}")
                LOG.info(f"MSSQL error dict: {e.__dict__}")
                span = tracer.current_span()
                span.set_status(status=status.Status.from_exception(e))
                raise
            finally:
                cur.close()

            return data

victoraugustolls on Oct 31, 2019