cluster-api: clusterctl init fails when existing cert-manager runs 1.0+

What steps did you take and what happened:

Already have an existing cluster where you want to install a cluster-api management cluster.

Already have cert-manager installed in said cluster:

$ kubectl api-resources | grep certmanager.k8s.io
certificaterequests               cr,crs             certmanager.k8s.io             true         CertificateRequest
certificates                      cert,certs         certmanager.k8s.io             true         Certificate
challenges                                           certmanager.k8s.io             true         Challenge
clusterissuers                                       certmanager.k8s.io             false        ClusterIssuer
issuers                                              certmanager.k8s.io             true         Issuer
orders                                               certmanager.k8s.io             true         Order

Already have cert-manager version 1.x+

$ kubectl explain certificaterequests
KIND:     CertificateRequest
VERSION:  cert-manager.io/v1

DESCRIPTION:
     <empty>

Run clusterctl version 0.3.10

$ asdf current clusterctl
clusterctl      0.3.10          ~/.tool-versions

$ asdf exec clusterctl version
clusterctl version: &version.Info{Major:"0", Minor:"3", GitVersion:"v0.3.10", GitCommit:"af6630920560ca0e12179897b96d6ea8bd830b63", GitTreeState:"clean", BuildDate:"2020-10-01T14:30:28Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"darwin/amd64"}

Try to install CAPI:

$ asdf exec clusterctl init --core cluster-api:v0.3.10 --bootstrap kubeadm:v0.3.10 --control-plane kubeadm:v0.3.10
Fetching providers
Installing cert-manager Version="v0.16.1"
Error: action failed after 10 attempts: failed to update cert-manager component apiextensions.k8s.io/v1beta1, Kind=CustomResourceDefinition, /certificaterequests.cert-manager.io: CustomResourceDefinition.apiextensions.k8s.io "certificaterequests.cert-manager.io" is invalid: status.storedVersions[0]: Invalid value: "v1": must appear in spec.versions

What did you expect to happen:

Having one or more of these options:

cert-manager not attempted to be installed (as it’s CRDs and controllers already exist in this cluster).
Being able to opt-out of installing cert-manager, as it’s already installed (as suggested in #3837).
cert-manager version embedded in cluster-api being upgraded to 1.0+ (as discussed in #3781).

Environment:

Cluster-api version: 0.3.10
Minikube/KIND version: N/A (using GKE)
Kubernetes version: (use kubectl version): Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.15", GitCommit:"2adc8d7091e89b6e3ca8d048140618ec89b39369", GitTreeState:"clean", BuildDate:"2020-09-02T11:40:00Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"darwin/amd64"} Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.15-gke.500", GitCommit:"f7db507aabec3b78cba0c27c616f4974213db6fd", GitTreeState:"clean", BuildDate:"2020-09-21T09:20:41Z", GoVersion:"go1.13.15b4", Compiler:"gc", Platform:"linux/amd64"}
OS (e.g. from /etc/os-release):

/kind bug /area clusterctl

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 2
Comments: 46 (29 by maintainers)

Most upvoted comments

Moving forward, in v1alpha4, I think we should allow users the possibility to opt put from clusterctl managing cert-manager.

I’m referring to #3837, whit a slight variation that opt in (implicit) and opt out (explicit) should apply both to init and upgrade

fabriziopandini on Nov 8, 2020

I will try to reproduce later, but I can’t assure you when. might be @wfernandes can help here

fabriziopandini on Oct 21, 2020

What about moving the issue to the cert-manager issue, given that as far as I understand the problems here seme more related to different cert-manager deployments. WRT to clusterctl, what we are doing in clustectl is exactly what is described in https://cert-manager.io/docs/installation/kubernetes/#verifying-the-installation, so as soon as you get an installation passing this procedure, clusterctl should use it if the version is newer that the one already installed.

Moving forward, in v1alpha4, I think we should allow users the possibility to opt put from clusterctl managing cert-manager.

fabriziopandini on Nov 6, 2020

Just to provide an update. I was able to reproduce the original error.

Thanks @MPV for the context via slack. I’ll update this thread with details once I can confirm the root cause. I’m pretty close with narrowing it down. 🙂

wfernandes on Oct 21, 2020