cert-manager: Error initializing issuer: context deadline exceeded
Describe the bug:
Failed to create clusterissuer, the following is the error log of cert-manager:
$ kubectl -n cert-manager logs -f cert-manager-59d959c87c-qsxbc
I0914 17:33:32.129086 1 controller.go:186] cert-manager/controller "msg"="starting controller" "controller"="clusterissuers"
I0914 17:33:32.129113 1 controller.go:163] cert-manager/controller "msg"="not starting controller as it's disabled" "controller"="certificatesigningrequests-issuer-venafi"
I0914 17:33:32.129130 1 controller.go:186] cert-manager/controller "msg"="starting controller" "controller"="certificates-revision-manager"
I0914 17:33:32.129143 1 controller.go:163] cert-manager/controller "msg"="not starting controller as it's disabled" "controller"="certificatesigningrequests-issuer-selfsigned"
I0914 17:33:32.129656 1 controller.go:186] cert-manager/controller "msg"="starting controller" "controller"="certificates-key-manager"
I0914 17:33:32.129744 1 controller.go:186] cert-manager/controller "msg"="starting controller" "controller"="orders"
I0914 17:33:32.133919 1 controller.go:186] cert-manager/controller "msg"="starting controller" "controller"="ingress-shim"
I0914 17:33:32.133980 1 controller.go:186] cert-manager/controller "msg"="starting controller" "controller"="certificaterequests-approver"
I0914 17:33:32.133996 1 controller.go:186] cert-manager/controller "msg"="starting controller" "controller"="certificaterequests-issuer-ca"
I0914 17:33:32.134022 1 controller.go:186] cert-manager/controller "msg"="starting controller" "controller"="certificates-issuing"
I0914 17:33:32.134044 1 controller.go:186] cert-manager/controller "msg"="starting controller" "controller"="certificaterequests-issuer-vault"
I0914 17:33:32.134078 1 controller.go:186] cert-manager/controller "msg"="starting controller" "controller"="certificates-readiness"
I0914 17:33:32.236494 1 controller.go:186] cert-manager/controller "msg"="starting controller" "controller"="challenges"
I0914 17:33:32.236658 1 controller.go:186] cert-manager/controller "msg"="starting controller" "controller"="certificates-trigger"
I0914 17:33:32.236691 1 controller.go:186] cert-manager/controller "msg"="starting controller" "controller"="certificates-request-manager"
I0914 17:35:53.546363 1 setup.go:219] cert-manager/controller/clusterissuers "msg"="ACME server URL host and ACME private key registration host differ. Re-checking ACME account registration" "related_resource_kind"="Secret" "related_resource_name"="letsencrypt-staging" "related_resource_namespace"="cert-manager" "resource_kind"="ClusterIssuer" "resource_name"="letsencrypt-staging" "resource_namespace"="" "resource_version"="v1"
E0914 17:36:03.547090 1 setup.go:259] cert-manager/controller/clusterissuers "msg"="failed to register an ACME account" "error"="context deadline exceeded" "related_resource_kind"="Secret" "related_resource_name"="letsencrypt-staging" "related_resource_namespace"="cert-manager" "resource_kind"="ClusterIssuer" "resource_name"="letsencrypt-staging" "resource_namespace"="" "resource_version"="v1"
I0914 17:36:03.547134 1 conditions.go:95] Setting lastTransitionTime for Issuer "letsencrypt-staging" condition "Ready" to 2021-09-14 17:36:03.547122659 +0800 CST m=+151.486801748
E0914 17:36:03.547165 1 sync.go:60] cert-manager/controller/clusterissuers "msg"="error setting up issuer" "error"="context deadline exceeded" "resource_kind"="ClusterIssuer" "resource_name"="letsencrypt-staging" "resource_namespace"="" "resource_version"="v1"
E0914 17:36:03.548101 1 controller.go:163] cert-manager/controller/clusterissuers "msg"="re-queuing item due to error processing" "error"="context deadline exceeded" "key"="letsencrypt-staging"
I0914 17:36:08.549436 1 setup.go:219] cert-manager/controller/clusterissuers "msg"="ACME server URL host and ACME private key registration host differ. Re-checking ACME account registration" "related_resource_kind"="Secret" "related_resource_name"="letsencrypt-staging" "related_resource_namespace"="cert-manager" "resource_kind"="ClusterIssuer" "resource_name"="letsencrypt-staging" "resource_namespace"="" "resource_version"="v1"
E0914 17:36:18.550061 1 setup.go:259] cert-manager/controller/clusterissuers "msg"="failed to register an ACME account" "error"="context deadline exceeded" "related_resource_kind"="Secret" "related_resource_name"="letsencrypt-staging" "related_resource_namespace"="cert-manager" "resource_kind"="ClusterIssuer" "resource_name"="letsencrypt-staging" "resource_namespace"="" "resource_version"="v1"
I0914 17:36:18.550114 1 conditions.go:95] Setting lastTransitionTime for Issuer "letsencrypt-staging" condition "Ready" to 2021-09-14 17:36:18.550108288 +0800 CST m=+166.489787378
E0914 17:36:18.550143 1 sync.go:60] cert-manager/controller/clusterissuers "msg"="error setting up issuer" "error"="context deadline exceeded" "resource_kind"="ClusterIssuer" "resource_name"="letsencrypt-staging" "resource_namespace"="" "resource_version"="v1"
E0914 17:36:18.550260 1 controller.go:163] cert-manager/controller/clusterissuers "msg"="re-queuing item due to error processing" "error"="context deadline exceeded" "key"="letsencrypt-staging"
I0914 17:36:28.551558 1 setup.go:219] cert-manager/controller/clusterissuers "msg"="ACME server URL host and ACME private key registration host differ. Re-checking ACME account registration" "related_resource_kind"="Secret" "related_resource_name"="letsencrypt-staging" "related_resource_namespace"="cert-manager" "resource_kind"="ClusterIssuer" "resource_name"="letsencrypt-staging" "resource_namespace"="" "resource_version"="v1"
$ kubectl -n cert-manager logs -f cert-manager-cainjector-6ff65c66fd-z8tbw
I0915 10:48:50.725518 1 controller.go:178] cert-manager/secret/validatingwebhookconfiguration/generic-inject-reconciler "msg"="updated object" "resource_kind"="ValidatingWebhookConfiguration" "resource_name"="cert-manager-webhook" "resource_namespace"="" "resource_version"="v1"
E0915 10:48:50.733174 1 controller.go:175] cert-manager/secret/validatingwebhookconfiguration/generic-inject-reconciler "msg"="unable to update target object with new CA data" "error"="Operation cannot be fulfilled on validatingwebhookconfigurations.admissionregistration.k8s.io \"cert-manager-webhook\": the object has been modified; please apply your changes to the latest version and try again" "resource_kind"="ValidatingWebhookConfiguration" "resource_name"="cert-manager-webhook" "resource_namespace"="" "resource_version"="v1"
E0915 10:48:50.733316 1 controller.go:304] cert-manager/secret/validatingwebhookconfiguration/controller/controller-for-secret-validatingwebhookconfiguration "msg"="Reconciler error" "error"="Operation cannot be fulfilled on validatingwebhookconfigurations.admissionregistration.k8s.io \"cert-manager-webhook\": the object has been modified; please apply your changes to the latest version and try again" "name"="cert-manager-webhook" "namespace"=""
I0915 10:48:50.736253 1 controller.go:178] cert-manager/secret/mutatingwebhookconfiguration/generic-inject-reconciler "msg"="updated object" "resource_kind"="MutatingWebhookConfiguration" "resource_name"="cert-manager-webhook" "resource_namespace"="" "resource_version"="v1"
I0915 10:48:50.741147 1 controller.go:178] cert-manager/secret/validatingwebhookconfiguration/generic-inject-reconciler "msg"="updated object" "resource_kind"="ValidatingWebhookConfiguration" "resource_name"="cert-manager-webhook" "resource_namespace"="" "resource_version"="v1"
$ kubectl -n cert-manager logs -f cert-manager-webhook-6984c6cbbc-zhwjw
W0915 10:48:33.783799 1 client_config.go:615] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
W0915 10:48:33.785307 1 client_config.go:615] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
I0915 10:48:33.785502 1 webhook.go:70] cert-manager/webhook "msg"="using dynamic certificate generating using CA stored in Secret resource" "secret_name"="cert-manager-webhook-ca" "secret_namespace"="cert-manager"
I0915 10:48:33.788419 1 server.go:138] cert-manager/webhook "msg"="listening for insecure healthz connections" "address"=":6080"
I0915 10:48:33.788536 1 server.go:169] cert-manager/webhook "msg"="listening for secure connections" "address"=":10250"
I0915 10:48:33.789029 1 server.go:201] cert-manager/webhook "msg"="registered pprof handlers"
I0915 10:48:34.819424 1 dynamic_source.go:272] cert-manager/webhook "msg"="Updated serving TLS certificate"
Anything else we need to know?:
cat staging-clusterissuer.yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
# The ACME server URL
server: https://acme-staging-v02.api.letsencrypt.org/directory
# Email address used for ACME registration
email: admin@example.com
# Name of a secret used to store the ACME account private key
privateKeySecretRef:
name: letsencrypt-staging
# Enable the HTTP-01 challenge provider
solvers:
- http01:
ingress:
class: nginx
$ kubectl create -f staging-clusterissuer.yaml
clusterissuer.cert-manager.io/letsencrypt-staging created
$ kubectl -n cert-manager describe clusterissuer letsencrypt-staging
Name: letsencrypt-staging
Namespace:
Labels: <none>
Annotations: <none>
API Version: cert-manager.io/v1
Kind: ClusterIssuer
Metadata:
Creation Timestamp: 2021-09-14T09:47:49Z
Generation: 1
Managed Fields:
API Version: cert-manager.io/v1
Fields Type: FieldsV1
fieldsV1:
f:spec:
.:
f:acme:
.:
f:email:
f:privateKeySecretRef:
.:
f:name:
f:server:
f:solvers:
Manager: kubectl
Operation: Update
Time: 2021-09-14T09:47:49Z
Resource Version: 270254
Self Link: /apis/cert-manager.io/v1/clusterissuers/letsencrypt-staging
UID: dcb7e1b4-145e-4a3f-9bca-d663738699c9
Spec:
Acme:
Email: admin@example.com
Preferred Chain:
Private Key Secret Ref:
Name: letsencrypt-staging
Server: https://acme-staging-v02.api.letsencrypt.org/directory
Solvers:
http01:
Ingress:
Class: nginx
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning ErrInitIssuer 83s (x7 over 7m38s) cert-manager Error initializing issuer: context deadline exceeded
Environment details::
- Kubernetes version: v1.18.18
- cert-manager version: 1.5.3
- Install method: static manifests
/kind bug
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 25 (3 by maintainers)
We’ve now published cert-manager 1.8.2 and 1.7.3 which include these fixes. @omBratteng mentioned on slack that they tested and it worked as expected with Sectigo!
Thanks very much to everyone who was involved in this issue - I’ll close it now as it seems to be completed ❤️
This should be fixed by #5226 which is backported into cert-manager 1.8 and 1.7 by #5231 and #5232.
We should hopefully be able to release new versions of cert-manager with these fixes soon, either today, tomorrow or at the start of next week 👍
same issue with zerossl:
Error initializing issuer: context deadline exceededEnvironment details:: Kubernetes version: v1.21.4 cert-manager version: 1.8.0 Install method: helm chart
I’m running into the same issue with ZeroSSL.
Related #5080 #5108?
Yes, a configureable context deadline would solve it.
I will verify the actual duration it took tomorrow, but I think it was 25 seconds.
@hadogenes can you please tell us what exactly you corrected on your DNS config ?