cert-manager: secret not found when using the cluster issuer

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:

/kind bug

/kind feature

What happened: When switching from Issuer to ClusterIssuer

Re-queuing item “" due to error processing: error getting clouddns service account: secret "” not found

What you expected to happen: A certificate to be issued

How to reproduce it (as minimally and precisely as possible): following the docs and then migrate the to using ClusterIssuers

Anything else we need to know?: the secret existed in both the certificates namespace and in the default namespace

Environment:

  • Kubernetes version (use kubectl version):
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.2", GitCommit:"81753b10df112992bf51bbc2c2f85208aad78335", GitTreeState:"clean", BuildDate:"2018-05-12T04:12:12Z", GoVersion:"go1.9.6", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"9+", GitVersion:"v1.9.6-gke.1", GitCommit:"cb151369f60073317da686a6ce7de36abe2bda8d", GitTreeState:"clean", BuildDate:"2018-04-07T22:06:59Z", GoVersion:"go1.9.3b4", Compiler:"gc", Platform:"linux/amd64"}

  • Cloud provider or hardware configuration**:
  • Install tools:
  • Others:

config

---
# Source: cert-manager/templates/00-namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: "cert-manager"

---
# Source: cert-manager/templates/certificate-crd.yaml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: certificates.certmanager.k8s.io
  labels:
    app: cert-manager
    chart: cert-manager-v0.3.2
    release: cert-manager
    heritage: Tiller
spec:
  group: certmanager.k8s.io
  version: v1alpha1
  scope: Namespaced
  names:
    kind: Certificate
    plural: certificates
    shortNames:
      - cert
      - certs

---
# Source: cert-manager/templates/clusterissuer-crd.yaml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: clusterissuers.certmanager.k8s.io
  labels:
    app: cert-manager
    chart: cert-manager-v0.3.2
    release: cert-manager
    heritage: Tiller
spec:
  group: certmanager.k8s.io
  version: v1alpha1
  names:
    kind: ClusterIssuer
    plural: clusterissuers
  scope: Cluster
---
# Source: cert-manager/templates/issuer-crd.yaml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: issuers.certmanager.k8s.io
  labels:
    app: cert-manager
    chart: cert-manager-v0.3.2
    release: cert-manager
    heritage: Tiller
spec:
  group: certmanager.k8s.io
  version: v1alpha1
  names:
    kind: Issuer
    plural: issuers
  scope: Namespaced
---
# Source: cert-manager/templates/deployment.yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: cert-manager
  namespace: "cert-manager"
  labels:
    app: cert-manager
    chart: cert-manager-v0.3.2
    release: cert-manager
    heritage: Tiller
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cert-manager
      release: cert-manager
  template:
    metadata:
      labels:
        app: cert-manager
        release: cert-manager
      annotations:
    spec:
      serviceAccountName: default
      containers:
        - name: cert-manager
          image: "quay.io/jetstack/cert-manager-controller:v0.3.0"
          imagePullPolicy: IfNotPresent
          args:
          - --cluster-resource-namespace=$(POD_NAMESPACE)
          - --leader-election-namespace=$(POD_NAMESPACE)
          env:
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          resources:
            requests:
              cpu: 10m
              memory: 32Mi
---
# this secret is present in both default and "somenamespacehere"
apiVersion: v1
kind: Secret
metadata:
  name: clouddns-service-account
type: Opaque
data:
  service-account.json: *****
---
apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    email: blah@example.com

    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
      name: letsencrypt-staging

    # ACME DNS-01 provider configurations
    dns01:

      # Here we define a list of DNS-01 providers that can solve DNS challenges
      providers:

        - name: prod-dns
          clouddns:
            # A secretKeyRef to a google cloud json service account
            serviceAccountSecretRef:
              name: clouddns-service-account
              key: service-account.json
            # The project in which to update the DNS zone
            project: *****
---
apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
  name: ****
  namespace: somenamespacehere
spec:
  secretName: tls
  issuerRef:
    name: letsencrypt-staging
    kind: ClusterIssuer
  commonName: '*.example.com'
  dnsNames:
  - www.example.com
  acme:
    config:
    - dns01:
        provider: prod-dns
      domains:
      - '*.example.com'
      - www.example.com

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 2
  • Comments: 28 (2 by maintainers)

Commits related to this issue

Most upvoted comments

I found that the secret had to be in the same namespace as the cert-manager Service in order for it to work.

Can we reopen this issue as a docs request for explaining this important detail?

@darkyat thanks for the suggestion, unfortunately that had no effect. I also created the secret in that namespace too. the secret now exists in default, “somenamespacehere”, kube-system

However creating the secret in the namespace ‘cert-manager’ appears to have fixed the issue! Can the scope of this bug be to please update the documentation regarding this?

👍 on adding the cert-manager namespace to the documentation for secrets

I found that the secret had to be in the same namespace as the cert-manager Service in order for it to work.

Can we reopen this issue as a docs request for explaining this important detail?

+1. This bit me as well. The instructions tell to deploy cert-manager into “cert-manager” namespace: https://docs.cert-manager.io/en/latest/getting-started/install.html but the clouddns instruction does not mention that the namespace must be the same for the secret: https://docs.cert-manager.io/en/latest/tasks/acme/configuring-dns01/google.html

Added certificate into same namespace where deployed cert-manager and added CAA dns records to the root domain, and now it works

@MartianGreed (et al) thanks for raising the issue and apologies for not replying sooner.

Any Secret associated with a ClusterIssuer must reside in the --cluster-resource-namespace, which is the namespace cert-manager by default, but may be overridden by whoever installed your cert-manager.

See Cluster Resource Namespace and please help us improve that documentation if you have time.

Having the same issue - the ClusterIssuer and secret are all in the same namespace (kube-system), still getting the Error getting certificate 'ifconfig-tls': secret "ifconfig-tls" not found error. I’m wondering if something about cross namespace stuff is borking.

@perspectivus1 you do realise i replaced the actual domain name email address with example.com rather than their real counterparts?

Same problem here. The Digital Ocean documentation didn’t say anything about having to put the secret into a specific namespace for the ClusterIssuer.

Very old versions of cert-manager used to use kube-system as the cluster-resource namespace and your cert-manager administrator may have continued that practice:

I’m using ClusterIssuer with digitalocean and DNS01 challenge.

The API Key secret is only found when created in cert-manager namespace.

I could not find any mentioning of this fact in the current documentation.

I’ve installed cert-manager using Helm:

helm install \
  cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.5.3 \
  --set installCRDs=true

I had the same issue. Moving the secret to kube-system namespace solved it for me (was default).

Could you give a try to deploy your clusterissuer secret into the kube-system namespace, @mgazza?