kyverno: [Bug] CustomResourceDefinition.apiextensions.k8s.io "policies.kyverno.io" is invalid: metadata.annotations: Too long: must have at most 262144 bytes

Kyverno Version

1.8.0

Description

Updating the chart from version 2.5.3 to version 2.6.0 using ArgoCD the Kyverno chart produces this error:

Failed sync attempt to 4989723e84c7b4a4cca397b07c8aa3eec457a6ed: one or more objects failed to apply, reason: CustomResourceDefinition.apiextensions.k8s.io "clusterpolicies.kyverno.io" is invalid: metadata.annotations: Too long: must have at most 262144 bytes,CustomResourceDefinition.apiextensions.k8s.io "policies.kyverno.io" is invalid: metadata.annotations: Too long: must have at most 262144 bytes (retried 5 times).

This happens due to the new CRDs file which has become really massive. It went from the size of 346547 bytes to the size of 1278510 bytes. Almost 4 times bigger, 3.7 to be precise.

I would kindly ask you to split the CRDs file into two or more parts (with some criteria) so that these problems are not created during updates.

Slack discussion

No response

Troubleshooting

  • I have read and followed the documentation AND the troubleshooting guide.
  • I have searched other issues in this repository and mine is not recorded.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 1
  • Comments: 15 (13 by maintainers)

Most upvoted comments

We can get this in 1.8.2 🤞

Yes, we are aware of this issue, we documented it along with the fix here https://kyverno.io/docs/installation/#notes-for-argocd-users

I think this is the link needed for ArgoCD users: https://kyverno.io/docs/installation/platform-notes/#notes-for-argocd-users

Just in case others hit this and use spinnaker. The option appears to be to add the strategy.spinnaker.io/replace annotation to all resources you want spinnaker to use replace for instead of apply

https://spinnaker.io/docs/reference/providers/kubernetes-v2/

As of Spinnaker 1.14, you can force Spinnaker to use replace instead of of apply while deploying a Kubernetes resource. This may be useful for resources such as ConfigMap which may exceed the annotation size limit of 262144 characters.

When set to 'true' for a versioned resource, this will update your resources using replace. Refer to [Kubernetes Object Management](https://kubernetes.io/docs/concepts/overview/object-management-kubectl/overview/#imperative-object-configuration) for more details on object configuration and trade-offs.

The default behavior is 'false'.

So I guess we have a couple possible options:

  • fork the CRDs and manage them ourselves going forward by adding the annotation and set installCRDs: false - might work.
  • contribute / suggest adding option to the Helm chart to allow adding optional annotations to the CRDs so spinnaker knows to run replace. I think I might open an issue to see if the project/community is open to this solution.

Closing as the documentation is available and provides a workaround. Please reopen if this needs to be discussed further.