pipeline: Publish task fails, IMAGES result too large
Expected Behavior
It is possible to release Tekton Pipelines
Actual Behavior
The publish task fails because the IMAGES result it too large:
{"level":"fatal","ts":1633523339.3185616,"caller":"entrypoint/entrypointer.go:203","msg":"Error while handling results: Termination message is above max allowed size 4096, caused by large task result.","stacktrace":"github.com/tektoncd/pipeline/pkg/entrypoint.Entrypointer.Go\n\tgithub.com/tektoncd/pipeline/pkg/entrypoint/entrypointer.go:203\nmain.main\n\tgithub.com/tektoncd/pipeline/cmd/entrypoint/main.go:126\nruntime.main\n\truntime/proc.go:225"}
Steps to Reproduce the Problem
- Trigger a nightly release
Additional Info
The IMAGES result is used by Tekton Chais to sign the container images.
The result includes all the container images produced by ko plus all their copies to the various regional registries.
- Kubernetes version:
Server Version: version.Info{Major:"1", Minor:"19+", GitVersion:"v1.19.13-gke.1900", GitCommit:"ee714a7b695ca42b9bd0c8fe2c0159024cdcba5e", GitTreeState:"clean", BuildDate:"2021-08-11T09:19:42Z", GoVersion:"go1.15.13b5", Compiler:"gc", Platform:"linux/amd64"}
- Tekton Pipeline version:
Client version: 0.19.0
Pipeline version: v0.27.3
Triggers version: v0.16.0
Dashboard version: v0.19.0
About this issue
- Original URL
- State: open
- Created 3 years ago
- Comments: 19 (15 by maintainers)
Commits related to this issue
- Only add gcr.io images to publish results The release process generates and number of container images and publishes them to gcr.io as well as three regional versions of the registry. Today all those... — committed to afrittoli/pipeline by afrittoli 3 years ago
- Only add gcr.io images to publish results The release process generates and number of container images and publishes them to gcr.io as well as three regional versions of the registry. Today all those... — committed to tektoncd/pipeline by afrittoli 3 years ago
- Only add gcr.io images to publish results The release process generates and number of container images and publishes them to gcr.io as well as three regional versions of the registry. Today all those... — committed to chenbh/pipeline by afrittoli 3 years ago
Successful nightly run: https://dashboard.dogfooding.tekton.dev/#/namespaces/tekton-nightly/pipelineruns/pipeline-release-nightly-gp4wm?pipelineTask=git-clone&step=clone
The result looks like this:
That is 4572 characters, which won’t fit. I think the only alternative for now is to only sign the image on
gcr.ioand we can start signing the geo copies once we solve the issue on results size.Thanks @wlynch - good point, I agree we should not sign regional copies separately.
Since the signing happens out of band (performed by chain) we cannot really copy the signature to the regional copies, unless we trigger another pipeline after the signature happen. This is probably ok since signature files are much smaller than the images.
We could copy the SBOM files around, but that’s a separate issue. I would propose we close this one.
Yes, indeed. We store results in the POD termination message, so having multiple results or multiple steps does not help.
Heh, indeed… but we’ll need a solution before the TEP though. Using multiple results would not help, we would need to use multiple tasks 😅