cert-manager: Error from server: conversion webhook

I also fall into this issue and was able to recover successfully. How this situation came at my end? We had to setup new cluster, after taking backup of certificates, which were issued by cert manager.

As per the cert manager I took backup. At the time of backup, my kubernetes version was v1.18.18 and cert manager version was v0.16.1(https://github.com/jetstack/cert-manager/releases/download/v0.16.1/cert-manager.yaml).

After setting up new cluster, what I did wrong? I installed again cert manager v0.16.1 and restored from backup. Certificate restore was not working, so that we thought of upgrading cert manager version v1.3.1 without any conversion of existing certificates. That did not helped in restoring the certificate and we uninstall and re-installed v0.16.1. That brang the disaster. Now re-installation was not working( may be we did some extra steps as well, we dont remember). We tried several uninstall and reinstall for v.0.16.1 and v1.3.1 and kubectl commands for getting ClusterIssuer was not working and as well as was not working for getting certificates list. System was total broken.

Reason was that CRD : certificaterequests.cert-manager.io was being shown as"Deletion in progress", which could not get completed as cert manager hook container was no more. We installed again the cert manager v.1.3.1 and found that cert manager hook container was up but now it was giving SSL certificate error while deleting above CRDs.

This tells the whole sequence of event, how things went bad. We found helm did the trick afer patching the cert managers CRDS.

Following are step, one should try to recover:

1. Uninstall the existing cert manager version(if not done): kubectl delete -f https://github.com/jetstack/cert-manager/releases/download/vX.Y.Z/cert-manager.yaml, if you had tried multiple version of cert manager, then do it for all and change vX.Y.Z accordingly.

2. Get cert managers CRDs and patch. In my case, there were 3 CRDs, which were remaing event after install. kubectl patch crd/clusterissuers.cert-manager.io -p ‘{“metadata”:{“finalizers”:[]}}’ --type=merge kubectl patch crd/orders.acme.cert-manager.io -p ‘{“metadata”:{“finalizers”:[]}}’ --type=merge kubectl patch crd/certificaterequests.cert-manager.io -p ‘{“metadata”:{“finalizers”:[]}}’ --type=merge

3. Dont install with regular manifest i.e. using kubectl -f apply, because you had already failed , right? At least in my case.

4. Install with helm( ensure helm version must be v3 and above). Note we used cert manager v1.3.1 version(which is latest as on date today) #Install cert manager using helm. #!/bin/bash NAME_SPACE=cert-manager kubectl create namespace $NAME_SPACE helm repo add jetstack https://charts.jetstack.io helm repo update helm install cert-manager jetstack/cert-manager --namespace $NAME_SPACE --create-namespace --version v1.3.1 --set installCRDs=true

This is alll and it worked magically after spending 5 hours to recover broken cert manager. It worked like charm

Some body suggested changing the namespace, I did not try that, Hope above only steps are enough to recover broken cert manager.

Alright, after I fixed crds - getting this error now - conversion webhook for cert-manager.io/v1alpha3, Kind=ClusterIssuerList failed: Post https://cert-manager-webhook.platform.svc:443/convert?timeout=30s: x509: certificate signed by unknown authority

Is there a way to add alternative names to autogenerate cert for webhook? Thanks

I’ve killed my production cluster even with the same namespace install… Glad I had backups. Now keeping the old version.

Issue still exists with 1.0.4. Possible cause is: kubectl apply --validate=false if you do that, you can never ever install CRD again 😦 RIP

Edit: Solution is still to replace the hardcoded stuff in cert-manager.crds.yaml: https://github.com/jetstack/cert-manager/issues/2752#issuecomment-608357457

Make sure you edit the correct cert-manager.crds.yaml file!

The v0.15.0-alpha.0 release should not suffer from this issue and you should be able to get it installed into whatever namespace using --set installCRDs=true --namespace=<custom-namespace-name>

Please give it a try, and let us know if you run into any issues 😄

When you change cert-manager-webhook.cert-manager.svc to cert-manager-webhook.{{ Namespace }}.svc in the yaml it works

There are other spaces where you need to change cert-manager to the name of your namespace. Just search and replace everything you find with ‘namespace: cert-manager’