cloud-sql-proxy: Connection test occasionally causes TLS handshake failed errors
Bug Description
gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.8.0 and gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.8.1 takes 30s to establish a connection when using --run-connection-test. This delay intermittently introduces proxy connection errors.
Stacktrace
Version v2.8.1
2024/01/08 23:07:43 Authorizing with OAuth2 token
2024/01/08 23:07:44 [<project>:<region>:<instance>] Listening on [::]:5432
2024/01/08 23:07:44 The proxy has started successfully and is ready for new connections!
2024/01/08 23:07:44 Connection test started
2024/01/08 23:08:14 Connection test passed
2024/01/08 23:08:14 [<project>:<region>:<instance>] accepted connection from 192.168.192.3:35240
2024/01/08 23:08:44 [<project>:<region>:<instance>] failed to connect to instance: Dial error: handshake failed (connection name = "<project>:<region>:<instance>"): context deadline exceeded
pg_dump: error: connection to database "<databse>" failed: connection to server at "pg_dump--proxy" (192.168.192.2), port 5432 failed: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
Version v2.7.2
2024/01/09 19:24:00 Authorizing with OAuth2 token
2024/01/09 19:24:00 [<project>:<region>:<instance>] Listening on [::]:5432
2024/01/09 19:24:00 The proxy has started successfully and is ready for new connections!
2024/01/09 19:24:00 Connection test started
2024/01/09 19:24:00 Connection test passed
2024/01/09 19:24:01 [<project>:<region:<instance>] accepted connection from 172.23.0.3:60642
Steps to reproduce?
Start a proxy container pointing to an instance, run the connection test as part of startup
docker run --rm gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.8.0 <project>:<region>:<instance> --run-connection-test
Environment
- OS type and version:
uname -a
Linux 92671f21852e 6.4.16-linuxkit #1 SMP PREEMPT Wed Oct 25 16:32:24 UTC 2023 x86_64 GNU/Linux
- Cloud SQL Proxy version (
./cloud-sql-proxy --version):2.8.1+container - Proxy invocation command (for example,
./cloud-sql-proxy --port 5432 INSTANCE_CONNECTION_NAME):
Additional Details
No response
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 16 (10 by maintainers)
Commits related to this issue
- [Airflow] Use Workload separation for Kubernetes pod tasks (Recidiviz/recidiviz-data#35233) ## Description of the change Unreverts Recidiviz/recidiviz-data#35056 Upon deployment of the previous imp... — committed to Recidiviz/pulse-data by ohaibbq 2 years ago
- [Airflow] Use Workload separation for Kubernetes pod tasks (Recidiviz/recidiviz-data#35233) ## Description of the change Unreverts Recidiviz/recidiviz-data#35056 Upon deployment of the previous imp... — committed to Recidiviz/pulse-data by ohaibbq 2 years ago
FWIW I’d expect the connection test to be only used when you’re uncertain if the network path is open. Otherwise, I wouldn’t use it regularly. In the meantime, we’ll sort out this bug.