crossplane: Crossplane Core Fails to Start up after upgrading from V1.7.0 to V1.13.2
What happened?
After upgrading from 1.7.0 to 1.13.2 the crossplane core doesn’t startup. We see error messages like
crossplane: error: core.initCommand.Run(): cannot initialize core: cannot apply crd: cannot patch object: CustomResourceDefinition.apiextensions.k8s.io "compositionrevisions.apiextensions.crossplane.io" is invalid: status.storedVersions[0]: Invalid value: "v1alpha1": must appear in spec.versions.
This is a canary upgrade from 1.7.0 to 1.13.2. In the dev environment, we upgraded from 1.7.0 to 1.12.2 and then got the fix release of 1.13.2 and everything works fine. What is happening in the canary is this upgrade is from 1.7.0 to 1.13.2 and again we see a similar error before the 1.13.2 fix.
Something to note here is we did not have compositionrevisions enabled before the upgrade.
Logs from init-container before upgrade and new init-container after upgrade are attached
How can we reproduce it?
It happened in the transient test clusters that are CI/CD pipelines. But should be reproducible by upgrading from 1.7.0 to 1.13.2.
What environment did it happen in?
- Crossplane version: 1.13.2
- Cloud provider: AWS, GCP
- Kubernetes version v1.25.10
- Os: Unavailable at this time
- Kernel: Unavailable at this time
About this issue
- Original URL
- State: open
- Created 10 months ago
- Comments: 23 (21 by maintainers)
I also believe it’s fair to expect the end user to take an intermediary upgrade path. I do think we should add an upgrade section to the docs and make this expectation known to users: https://docs.crossplane.io/software/
cc @plumbis
Yes, @dee0sap, it should