node-postgres: Connection terminated unexpectedly only in Node.js v20
Has anyone encountered an issue migrating from Node.js v18 to v20?
I am trying isolate exactly what’s breaking, but so far unsuccessful.
All tests are passing in Node.js v18, but as soon as we try to run them in v20, they start to fail with:
FAIL test/contra-temporal-job-worker/workflows/generatePDFInvoice.test.ts > generatePDFInvoice for custom UNPAID invoice after milestone autogenerated invoice
Error: Connection terminated unexpectedly
❯ Connection.<anonymous> ../../node_modules/.pnpm/pg@8.11.3/node_modules/pg/lib/client.js:132:73
❯ Object.onceWrapper node:events:628:28
❯ Connection.emit node:events:514:28
❯ Connection.emit node:domain:488:12
❯ Socket.<anonymous> ../../node_modules/.pnpm/pg@8.11.3/node_modules/pg/lib/connection.js:63:12
❯ Socket.emit node:events:514:28
❯ Socket.emit node:domain:488:12
❯ TCP.<anonymous> node:net:337:12
It happens to about ~1% of all tests (out of about a thousand).
It is not happening to the same tests.
Only happens when running a large number of parallel tests.
About this issue
- Original URL
- State: closed
- Created 8 months ago
- Comments: 16 (14 by maintainers)
I am having the exact same error here. Sometimes is when reaching out to a Redis instance and sometimes is when I have to reach out the DB itself (Mysql). Could not find a way to solve this without downgrade NodeJs version. It seems that is something with node indeed.
The node client crashing when the database fast shutdowns is a known thing: https://github.com/brianc/node-postgres/issues/1324#issuecomment-623311914
Without an error event handler, the client will crash the node process if the connection gets killed. That’s only if the connection is checked out as the pool attaches error handlers, but they’re only in place while the connection is in the pool. To handle all errant situations, you need to add error handlers as the connection crashed error comes as an event, not in response to a promise rejection or callback.
Maybe this is purely a race condition in the tests with the newer node versions running faster (or slower?) and that’s triggering that test case at the exact moment when the DB is being dropped.