sqlalchemy: BrokenPipeError or struct.error does not invalidate connection in SQLAlchemy

We are using pg8000 with SQLAlchemy in a docker environment. Everything works fine but while testing the application, whenever I restart the database(Postgresql) to simulate some error, the flask application cannot connect to the database. I’m getting either BrokenPipeError or struct.error. I found in the README of pg8000 this behaviour is mentioned. We tried to use SQLAlchemy’s pre-pinging to deal with disconnects. But raising these exceptions seems to not invalidate the connection. Here is a sample log

2021-03-16 13:55:19,928 DEBUG sqlalchemy.pool.impl.QueuePool Connection <pg8000.legacy.Connection object at 0x7ff0bbac4150> checked out from pool
2021-03-16 13:55:19,928 DEBUG sqlalchemy.pool.impl.QueuePool Pool pre-ping on connection <pg8000.legacy.Connection object at 0x7ff0bbac4150>
[2021-03-16 13:55:19,929] ERROR in app: Exception on /namespaces/ [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.7/site-packages/flask_restx/api.py", line 375, in wrapper
    resp = resource(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask_restx/resource.py", line 44, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask_httpauth.py", line 382, in decorated
    )(f)(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask_httpauth.py", line 149, in decorated
    user = self.authenticate(auth, password)
  File "/usr/local/lib/python3.7/site-packages/flask_httpauth.py", line 220, in authenticate
    return self.verify_password_callback(username, client_password)
  File "./core/model.py", line 104, in verify_password
    rci. Suspendisse ac accumsan ipsum. Suspendisse auctor venenatis magna
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2684, in first
    return self.limit(1)._iter().first()
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2771, in _iter
    execution_options={"_sa_orm_load_options": self.load_options},
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1652, in execute
    conn = self._connection_for_bind(bind, close_with_result=True)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1503, in _connection_for_bind
    engine, execution_options
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 738, in _connection_for_bind
    conn = bind.connect()
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 3066, in connect
    return self._connection_cls(self, close_with_result=close_with_result)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 91, in __init__
    else engine.raw_connection()
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 3145, in raw_connection
    return self._wrap_pool_connect(self.pool.connect, _connection)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 3112, in _wrap_pool_connect
    return fn()
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 301, in connect
    return _ConnectionFairy._checkout(self)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 781, in _checkout
    result = pool._dialect.do_ping(fairy.connection)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 648, in do_ping
    cursor.execute(self._dialect_specific_select_one)
  File "/usr/local/lib/python3.7/site-packages/pg8000/legacy.py", line 175, in execute
    self._c.execute_unnamed("begin transaction")
  File "/usr/local/lib/python3.7/site-packages/pg8000/core.py", line 735, in execute_unnamed
    self.handle_messages(context)
  File "/usr/local/lib/python3.7/site-packages/pg8000/core.py", line 895, in handle_messages
    code, data_len = ci_unpack(self._read(5))
struct.error: unpack_from requires a buffer of at least 5 bytes
2021-03-16 13:55:19,933 DEBUG sqlalchemy.pool.impl.QueuePool Connection <pg8000.legacy.Connection object at 0x7ff0bbac4150> being returned to pool
2021-03-16 13:55:19,933 DEBUG sqlalchemy.pool.impl.QueuePool Connection <pg8000.legacy.Connection object at 0x7ff0bbac4150> rollback-on-return

I created a minimal example to reproduce the issue.

git pull https://github.com/hbusul/sqlalchemy_pg800_pre_ping

docker-compose up --build

After 10 secs, from another terminal docker-compose restart database

I already opened a ticket in pg8000

I guess these exceptions should be wrapped in other exceptions. My question is that which exception would be suitable for the issue?

Versions

• OS: Linux
• Python: 3.7
• SQLAlchemy: 1.3.5
• Database: Postgresql
• DBAPI: pg8000?