rancher: helm upgrade is broken - certmanager.version is needed but can't be parsed.

Know this an old post but this might be a less hacky approach If anyone runs into this another solution that resolved it for us is using helm plugin mapkubeappis

helm plugin install https://github.com/helm/helm-mapkubeapis

create a map.yaml file with the troublesome api

mappings:
  - deprecatedAPI: "apiVersion: cert-manager.io/v1beta1\nkind: Issuer"
    newAPI: "apiVersion: cert-manager.io/v1\nkind: Issuer"
    deprecatedInVersion: "v1.9" #see next comment
    removedInVersion: "v1.16" #left default values may be optional didn't try seems innocuous

then upgrade the api example:

helm mapkubeapis rancher --namespace cattle-system --mapfile map.yaml

2022/02/05 21:12:36 Release ‘rancher’ will be checked for deprecated or removed Kubernetes APIs and will be updated if necessary to supported API versions. 2022/02/05 21:12:36 Get release ‘rancher’ latest version. 2022/02/05 21:12:37 Check release ‘rancher’ for deprecated or removed APIs… 2022/02/05 21:12:37 Found deprecated or removed Kubernetes API: “apiVersion: cert-manager.io/v1beta1 kind: Issuer” Supported API equivalent: “apiVersion: cert-manager.io/v1 kind: Issuer” 2022/02/05 21:12:37 Finished checking release ‘rancher’ for deprecated or removed APIs. 2022/02/05 21:12:37 Deprecated or removed APIs exist, updating release: rancher. 2022/02/05 21:12:37 Set status of release version ‘rancher.v6’ to ‘superseded’. 2022/02/05 21:12:37 Release version ‘rancher.v6’ updated successfully. 2022/02/05 21:12:37 Add release version ‘rancher.v7’ with updated supported APIs. 2022/02/05 21:12:38 Release version ‘rancher.v7’ added successfully. 2022/02/05 21:12:38 Release ‘rancher’ with deprecated or removed APIs updated successfully to new version. 2022/02/05 21:12:38 Map of release ‘rancher’ deprecated or removed APIs to supported versions, completed successfully.

Fix https://exploit.cz/rancher-upgrade-failed-cert-manager-io-v1beta1/

if you have certmanager.k8s.io/v1alpha1

replace cert-manager.io\/v1beta1 with certmanager.k8s.io\/v1alpha1 in step 3 in sed replace

Following up on my own post, I was able to resolve my upgrade issue (upgrading from Rancher 2.3 to 2.4 with Helm 3) by doing the following after getting the error:

Error: unable to build kubernetes objects from release manifest: unable to recognize "": no matches for kind "Issuer" in version "certmanager.k8s.io/v1alpha1"

I think this is essentially what @ebauman did. Step 1, download an old CRD with matching API: wget -O crds.yaml https://github.com/jetstack/cert-manager/releases/download/v0.10.1/cert-manager.yaml Step 2. Edit crds.yaml and remove all entries except the one that matches kind: Issuer Step 3. Apply the CRD: kubectl apply -f crds.yaml

At this point if you attempt to do the upgrade again with: helm upgrade rancher rancher-latest/rancher --namespace cattle-system --set hostname=xxxx --set certmanager.version="0.15" You may come across the following errors:

Error: UPGRADE FAILED: rendered manifests contain a resource that already exists. Unable to continue with update: Ingress "rancher" in namespace "cattle-system" exists and cannot be imported into the current release: invalid ownership metadata; label validation error: missing key "app.kubernetes.io/managed-by": must be set to "Helm"; annotation validation error: missing key "meta.helm.sh/release-name": must be set to "rancher"; annotation validation error: missing key "meta.helm.sh/release-namespace": must be set to "cattle-system"

To fix the above, edit the resource: kubectl get edit ingress rancher -n cattle-system and add the missing labels and annotations.

Error: UPGRADE FAILED: rendered manifests contain a resource that already exists. Unable to continue with update: Issuer "rancher" in namespace "cattle-system" exists and cannot be imported into the current release: invalid ownership metadata; label validation error: missing key
 "app.kubernetes.io/managed-by": must be set to "Helm"; annotation validation error: missing key "meta.helm.sh/release-name": must be set to "rancher"; annotation validation error: missing key "meta.helm.sh/release-namespace": must be set to "cattle-system"

To fix the above, edit the resource: kubectl edit issuer.cert-manager.io/rancher -n cattle-system and add the missing labels and annotations.

Now attempt to upgrade again with Helm, and it should work.

You may then need to remove the old CRDs: kubectl delete -f crds.yaml

In my case, I also had to edit the rancher ingress to use my letsencrypt production issuer.

I worked my way out of this, but it wasn’t pretty. First, to solve the missing CRD issue, I re-created the cert-manager CRDs using https://raw.githubusercontent.com/jetstack/cert-manager/v0.10.0/deploy/manifests/00-crds.yaml.

Then, I was faced with this error:

$ helm upgrade rancher rancher-latest/rancher --set hostname=my.rancher.com --set certmanager.version="0.12"
Error: UPGRADE FAILED: rendered manifests contain a resource that already exists. Unable to continue with update: Issuer "rancher" in namespace "cattle-system" exists and cannot be imported into the current release: invalid ownership metadata; annotation validation error: missing key "meta.helm.sh/release-name": must be set to "rancher"; annotation validation error: missing key "meta.helm.sh/release-namespace": must be set to "cattle-system"

I attemped to set those values on the issuer in question (issuer.cert-manager.io/rancher) but they did help matters. I ended up removing that issuer, and re-running helm upgrade. This resulted in a successful upgrade.