spegel: `verifyStatusResponse` does not work with containerd v2
Spegel version
v0.0.24
Kubernetes distribution
Talos
Kubernetes version
v1.31.0
CNI
Cilium
Describe the bug
After updating my homelab cluster to Talos v1.8.0-beta0, I noticed that the spegel pod no longer launched. The logs just say
{"time":"2024-09-10T07:29:25.425786168Z","level":"ERROR","source":{"function":"main.main","file":"/build/main.go","line":89},"msg":"run exit with error","err":"Containerd registry config path needs to be set for mirror configuration to take effect"}
before the program terminates. Looking at the code, that’s pkg.oci.containerd verifyStatusResponse() failing its check for ConfigPath in the received containerd config.
https://github.com/spegel-org/spegel/blob/main/pkg/oci/containerd.go#L126
I have a feeling Talos is not configuring containerd correctly, but the config file reads:
version = 3
[plugins]
[plugins.'io.containerd.cri.v1.images']
discard_unpacked_layers = false
[plugins.'io.containerd.cri.v1.images'.registry]
config_path = '/etc/cri/conf.d/hosts'
which according to https://github.com/containerd/containerd/blob/main/docs/cri/config.md is the correct way in 2.x.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 4
- Comments: 15 (9 by maintainers)
Commits related to this issue
- fix sepgel [issue5179](https://github.com/spegel-org/spegel/issues/579) — committed to soulwhisper/homelab-ops by soulwhisper 2 years ago
- fix sepgel https://github.com/spegel-org/spegel/issues/579 — committed to soulwhisper/homelab-ops by soulwhisper 2 years ago
@phillebaba any chance that PR can be looked at? Talos
1.8.0has packaged containerd2.0.0-rc.4.I have been busy with work lately so have not had time for Spegel. I would rather we fix the underlying problem rather than just ignore verification if the version is Containerd v2. I will have a look and see if this can be solved some other way.
I have spent some time looking at alternative methods of verifying the configuration without much luck. I have created an issue in Containerd https://github.com/containerd/containerd/issues/10780 for long term work. In the meantime I will work to merge #581 to disable verification for Containerd v2.
It would be nice to start testing Containerd v2 so Talos may be a good candidate to run e2e tests.
@aki263 for now I am using this image in the helm values
I have cherry-picked the patch of PR #581 into the v0.0.24 release tag and built the container image locally. Pushed it to my own registry and configured the helm chart to use my custom image instead. With the patch applied, spegel works perfectly again on Talos 1.8.0.