kubebuilder: kubebuilder make install fails

What broke? What’s expected?

Following the kubebuilder book, the make install step fails with the error message:

The CustomResourceDefinition "cronjobs.batch.tutorial.kubebuilder.io" is invalid: metadata.annotations: Too long: must have at most 262144 bytes

As described in this post the issue derives from the auto-generated annotation when running kubectl apply. Running the same command with kubectl create or kubectl replace works.

I was wondering how to correct this issue. Possible approaches:

easy, workaround: insert a warning into the book stating this problem and telling the user to edit the make install;
easy, workaround, fragile: make the install target slightly more intelligent and dynamically choose apply, create or replace depending on whether the CRDs exist and on the possible error;
harder: solve the problem on kubectl apply - here a further discussion needs to be triggered on whether we even want to solve this issue and what impacts that could generate (not sure it would be backwards compatible)
within kubectl apply make a more detailed error and suggest usage of create or replace commands instead.

Reproducing this issue

Follow the kubebuilder book up to the make install in this book section.

KubeBuilder (CLI) Version

Version: main.version{KubeBuilderVersion:“v3.10.0-110-gd9c14df9”, KubernetesVendor:“unknown”, GitCommit:“d9c14df9d84fafd08c9ec88441e9ce218ad6effa”, BuildDate:“2023-06-16T09:47:40Z”, GoOs:“darwin”, GoArch:“arm64”}

PROJECT version

version: “3”

Plugin versions

- go.kubebuilder.io/v4

Other versions

go version go1.20.4 darwin/arm64
sigs.k8s.io/controller-runtime v0.15.0

Extra Labels

/kind documentation

About this issue

Original URL
State: closed
Created a year ago
Reactions: 1
Comments: 17 (9 by maintainers)

Most upvoted comments

This is missing from cronjob main tutorial though and is also nor is it I believe mentioned in the book itself so far.

https://github.com/kubernetes-sigs/kubebuilder/blob/66f3fbc7ae349e8ee31f1c6eb8d55d5b1c78ce52/docs/book/src/cronjob-tutorial/testdata/project/Makefile#L47-L49

RafalSkolasinski on Jul 3, 2023

So would you see it as step instructing the reader of kubebuilder book to go and edit the Makefile just after it was generated by Kubebuilder to make sure it includes the crd:maxDescLen=0?

See the comment with the explanation:

# Note that the option maxDescLen=0 was added in the default scaffold in order to sort out the issue 
     # Too long: must have at most 262144 bytes. By using kubectl apply to create / update resources an annotation 
     # is created by K8s API to store the latest version of the resource ( kubectl.kubernetes.io/last-applied-configuration). 
     # However, it has a size limit and if the CRD is too big with so many long descriptions as this one it will cause the failure.

K8s has a limitation regards the size of CRDs. Please see: https://github.com/kubernetes-sigs/kubebuilder/issues/1140#issuecomment-1299307417

Therefore, what we can do here is:

a) Create a NOTE explaining the k8s size limitation
b) Explain that the workaround is to not kubectl apply when installing OR to set maxDescLen on controller-tools when generating the CRD
c) Explain that controller-gen introduce maxDescLen to work around this problem until we get rid of client-side apply
d) See the PR https://github.com/kubernetes-sigs/controller-tools/pull/536

When I say create a note? See the docs and look for <aside class="note">. Therefore we could add this info via a NOTE

<aside class="note">

<h1>To workaround the error `Too long: must have at most 262144 bytes`</h1>

Add: Why the error occur (k8s limitation)
Add: The workaround (use apply or maxDescLen)

</aside>

I met this problem too, and I tried to add crd:maxDescLen=0 but it didn’t work. So could you tell me what is the solution? Or did I miss something else?

@derecknowayback:

You need to add the option and then run make manifests to generate the CRD with. If remove the description of the CRDs did not sort out the problem and it is bigger than 1 GB then you need to check why to solve it.

camilamacedo86 on Jul 7, 2023

Hello @rbroggi @camilamacedo86 which would be the place to add this note. Should we add it under make install or in a faq section.

I’m guessing that what @camilamacedo86 had in mind was to have it close to the sections containing the shell commands where the simple kubectl apply embedded in the makefile fails.

rbroggi on Aug 19, 2023

HI @rbroggi,

If you look at the code generate for the sample of the tutorial the target to generate the manifest has a customization to avoid this scenario. I think we could add a note in the tutorial explaining it.

See: https://github.com/kubernetes-sigs/kubebuilder/blob/66f3fbc7ae349e8ee31f1c6eb8d55d5b1c78ce52/docs/book/src/multiversion-tutorial/testdata/project/Makefile#L41-L47

Would you like to contribute with this one?

camilamacedo86 on Jul 3, 2023