zitadel: Trouble deploying, Message=Instance not found

Hey all, was deploying this to see if it would be a good fit for out infrastructure and came up with an issue. I deployed it to kubernetes using the help chart and copied the values.yaml and modified it.

This is my values.yaml:

# Default values for zitadel.
zitadel:

  # The ZITADEL config under configmapConfig is written to a Kubernetes ConfigMap
  # See all defaults here:
  # https://github.com/zitadel/zitadel/blob/main/cmd/defaults.yaml
  configmapConfig:
    Log:
      Level: 'debug'
      Formatter:
        Format: text
    Database:
      cockroach:
        Host: "crdb-public"
        User:
          SSL:
            Mode: "verify-full"
        Admin:
          SSL:
            Mode: "verify-full"
    ExternalPort: 443
    ExternalDomain: "external.domain"
    ExternalSecure: true
    Machine:
      Identification:
        Hostname:
          Enabled: true
        Webhook:
          Enabled: false

  # The ZITADEL config under secretConfig is written to a Kubernetes Secret
  # See all defaults here:
  # https://github.com/zitadel/zitadel/blob/main/cmd/defaults.yaml
  secretConfig:
    Log:
      Level: 'debug'
      Formatter:
        Format: text
    Database:
      cockroach:
        User:
          Username: "db.username"
          Password: "db.password"
        Admin:
          Username: "root"
          Password: ""
    Metrics:
      Type: none
    Port: 8080
    ExternalPort: 443
    ExternalDomain: "external.domain"
    ExternalSecure: true
    TLS:
      Enabled: false
    Machine:
      Identification:
        Hostname:
          Enabled: true
        Webhook:
          Enabled: false
    SAML:
      Organisation:
        Name: "company.name"
        URL: "company.site"
      ContactPerson:
        ContactType: "technical"
        Company: "company.name"
        EmailAddress: "company.email"
    DefaultInstance:
      Org:
        Name: "name"
        Human:
          UserName: "user.name"
          Password: "user.password"
          Email: 
            Address: "company.email"
            Verified: true
      DomainPolicy:
        SMTPSenderAddressMatchesInstanceDomain: false
      SMTPConfiguration:
        SMTP:
          Host: "email.domain.name"
          User: "email.user.name"
          Password: "email.password"
        TLS: true
        From: "from.email"
        FromName: "from.name"

  # Reference the name of a secret that contains ZITADEL configuration.
  # The key should be named "config-yaml".
  configSecretName:

  # ZITADEL uses the masterkey for symmetric encryption.
  # You can generate it for example with tr -dc A-Za-z0-9 </dev/urandom | head -c 32
  masterkey: 'zitadel.masterkey'
  # Reference the name of the secret that contains the masterkey. The key should be named "zitadel-masterkey".
  # Note: Eighter zitadel.masterkey or zitadel.masterkeySecretName must be set
  masterkeySecretName: ""

  # The root CA Certificate needed for establishing secure database connections
  dbSslRootCrt: ''

  # The Secret containing the root CA Certificate at key ca.crt needed for establishing secure database connections
  dbSslRootCrtSecret: 'crdb-ca-secret'
  # dbSslRootCrtSecret: ''

  # The Secret containing the client CA Certificate and key at tls.crt and tls.key needed for establishing secure database connections
  dbSslClientCrtSecret: 'crdb-client-secret'
  # dbSslClientCrtSecret: ''

replicaCount: 1

image:
  repository: ghcr.io/zitadel/zitadel
  pullPolicy: IfNotPresent
  # Overrides the image tag whose default is the chart appVersion.
  tag: "v2.6.0"

chownImage:
  repository: alpine
  pullPolicy: IfNotPresent
  tag: "3.11"

imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""

serviceAccount:
  # Specifies whether a service account should be created
  create: true
  # Annotations to add to the service account
  annotations: {}
  # The name of the service account to use.
  # If not set and create is true, a name is generated using the fullname template
  name: ""

podAnnotations: {}

podSecurityContext:
  runAsNonRoot: true
  runAsUser: 1000

securityContext: {}

service:
  type: ClusterIP
  port: 8080
  protocol: http2
  annotations: {}

ingress:
  enabled: true
  className: "nginx"
  pathType: ImplementationSpecific
  path: /
  selfSigned: false
  hostname: external.domain
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
  hosts:
    - host: external.domain
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: tls.secret.name
      hosts: 
        - external.domain

resources: {}

nodeSelector: {}

tolerations: []

affinity: {}

metrics:
  enabled: false
  serviceMonitor:

    # If true, the chart creates a ServiceMonitor that is compatible with Prometheus Operator
    # https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api.md#monitoring.coreos.com/v1.ServiceMonitor.
    # The Prometheus community Helm chart installs this operator
    # https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack#kube-prometheus-stack
    enabled: false
    honorLabels: false
    honorTimestamps: true

cockroachdb:
  enabled: true
  fullnameOverride: crdb
  tls:
    enabled: true

Now trying to go to the external.domain/ui/console specified under ExternalDomain gives me a ID=QUERY-n0wng Message=Instance not found Error in browser. And a 404 without the /ui/console part.

Not sure how to troubleshoot this. No errors are raised in the logs for the pod/container. I did get the proper UI to load before when I was trying it out on non-https mode.

Also, the documentation is slightly skewed in three places from my perusal. In some places some of the values go in the configMap but it other places the same keys are using in the secretConfig instead.

The second place is that the documentation makes mention of a FirstInstance key, but the actual defaults file at: https://github.com/zitadel/zitadel/blob/main/cmd/defaults.yaml puts that information in the DefaultInstance key.

And the last place is that the documentation states: For more configuration options, go to the chart repo descriptions. with a link to the repo, but the repo only links you back to a different page of the docs which again, tells you that configuration details are described in the repo.

Don’t know if this was by intent but seems to just be a circular loop that doesn’t contain any detailed info for production deployment.