amazon-redshift-python-driver: Driver is unable to handle response from RedShift

Driver version

2.0.903

Redshift version

PostgreSQL 8.0.2 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.4.2 20041017 (Red Hat 3.4.2-6.fc3), Redshift 1.0.49676

Client Operating System

AWS Lambda (Amazon Linux)

Python version

3.10

Table schema

Doesn’t matter

Problem description

Getting UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9d in position 122: invalid start byte when simply connecting to RedShift. THIS IS VERY RARE OCCURENCE, in our case - 7 times out of 5000 invocations. I’m guessing, that RedShift is too busy and driver is unable to handle such event

Python Driver trace logs

  self.connection = self.engine.connect()
  File "/var/task/sqlalchemy/engine/base.py", line 3204, in connect
    return self._connection_cls(self, close_with_result=close_with_result)
  File "/var/task/sqlalchemy/engine/base.py", line 96, in __init__
    else engine.raw_connection()
  File "/var/task/sqlalchemy/engine/base.py", line 3283, in raw_connection
    return self._wrap_pool_connect(self.pool.connect, _connection)
  File "/var/task/sqlalchemy/engine/base.py", line 3250, in _wrap_pool_connect
    return fn()
  File "/var/task/sqlalchemy/pool/base.py", line 310, in connect
    return _ConnectionFairy._checkout(self)
  File "/var/task/sqlalchemy/pool/base.py", line 868, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/var/task/sqlalchemy/pool/base.py", line 476, in checkout
    rec = pool._do_get()
  File "/var/task/sqlalchemy/pool/impl.py", line 256, in _do_get
    return self._create_connection()
  File "/var/task/sqlalchemy/pool/base.py", line 256, in _create_connection
    return _ConnectionRecord(self)
  File "/var/task/sqlalchemy/pool/base.py", line 371, in __init__
    self.__connect()
  File "/var/task/sqlalchemy/pool/base.py", line 677, in __connect
    pool.dispatch.connect.for_modify(
  File "/var/task/sqlalchemy/event/attr.py", line 329, in _exec_w_sync_on_first_run
    self(*args, **kw)
  File "/var/task/sqlalchemy/event/attr.py", line 343, in __call__
    fn(*args, **kw)
  File "/var/task/sqlalchemy/util/langhelpers.py", line 1691, in go
    return once_fn(*arg, **kw)
  File "/var/task/sqlalchemy/engine/create.py", line 686, in first_connect
    dialect.initialize(c)
  File "/var/task/sqlalchemy/dialects/postgresql/base.py", line 3241, in initialize
    super(PGDialect, self).initialize(connection)
  File "/var/task/sqlalchemy/engine/default.py", line 407, in initialize
    self.server_version_info = self._get_server_version_info(
  File "/var/task/sqlalchemy/dialects/postgresql/base.py", line 3487, in _get_server_version_info
    v = connection.exec_driver_sql("select pg_catalog.version()").scalar()
  File "/var/task/sqlalchemy/engine/base.py", line 1669, in exec_driver_sql
    return self._exec_driver_sql(
  File "/var/task/sqlalchemy/engine/base.py", line 1578, in _exec_driver_sql
    ret = self._execute_context(
  File "/var/task/sqlalchemy/engine/base.py", line 1845, in _execute_context
    self._handle_dbapi_exception(
  File "/var/task/sqlalchemy/engine/base.py", line 2030, in _handle_dbapi_exception
    util.raise_(exc_info[1], with_traceback=exc_info[2])
  File "/var/task/sqlalchemy/util/compat.py", line 207, in raise_
    raise exception
  File "/var/task/sqlalchemy/engine/base.py", line 1802, in _execute_context
    self.dialect.do_execute(
  File "/var/task/sqlalchemy/engine/default.py", line 732, in do_execute
    cursor.execute(statement, parameters)
  File "/var/task/redshift_connector/cursor.py", line 211, in execute
    self._c.execute(self, operation, args)
  File "/var/task/redshift_connector/core.py", line 1279, in execute
    self.handle_messages(cursor)
  File "/var/task/redshift_connector/core.py", line 1359, in handle_messages
    self.message_types[code](self._read(data_len - 4), cursor)
  File "/var/task/redshift_connector/core.py", line 1350, in handle_DATA_ROW
    row.append(desc[0](data, data_idx, vlen))
  File "/var/task/redshift_connector/utils/type_utils.py", line 131, in text_recv
    return str(data[offset : offset + length], _client_encoding)