zitadel-charts: [Bug]: Zitadel deployment on EKS with AWS PostgreSQL RDS instance - setup job failure

Preflight Checklist

  • I could not find a solution in the documentation, the existing issues or discussions
  • I have joined the ZITADEL chat

Environment

Self-hosted

Version

8.5.0

Database

PostgreSQL

Database Version

16

Describe the problem caused by this bug

Attempting installation of Zitadel on EKS Kubernetes cluster fails at setup job when using AWS RDS for PostgreSQL. Initially the issue was with SSL, but after setting SSL mode to disable, and rds.force_ssl = 0 on RDS, I was able to connect successfully to RDS using username and password.

Database initialization was also successful, but setup job will get stuck to infinity with the following error:

time="2024-10-16T18:28:34Z" level=warning msg="migration already started, will check again in 5 seconds" caller="/home/runner/work/zitadel/zitadel/internal/migration/migration.go:130" migration step=projections.login_policies5

Can you help point out if this is known issue and there is a workaround.

To reproduce

  1. Provision PostgreSQL DB instance on RDS
  2. Edit Parameter Group and set rds.force_ssl = 0
  3. Create a secret with RDS DB credentials - host and password
  4. Deploy Zitadel on Kubernetes using Helm, while using correct values for configSecretName and configSecretKey
  5. Zitadel init job will be successful
  6. Setup job will not be successful

Screenshots

Image

Expected behavior

Init and Setup job successful for Zitadel to be healthy and running.

Operating System

Kubernetes (AWS EKS) - 1.30

Relevant Configuration

Database: Postgres: Port: 5432 Database: <rds-db-endpoint> MaxOpenConns: 20 MaxIdleConns: 10 MaxConnLifetime: 30m MaxConnIdleTime: 5m User: Username: zitadel SSL: Mode: disable Admin: Username: zitadel SSL: Mode: disable configSecretName: zitadel-config-secret configSecretKey: config.yaml

Additional Context

No response

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Reactions: 1
  • Comments: 18 (5 by maintainers)

Most upvoted comments

@jmutai As I mentioned the trick is to keep the default sslMode for RDS but ensure that Zitadel uses require

These are the envvars that I source

ZITADEL_DATABASE_POSTGRES_USER_SSL_MODE=require
ZITADEL_DATABASE_POSTGRES_ADMIN_SSL_MODE=require

And here is the working code (when I was using RDS).