spegel: Spegel pod gets into CrashLoopBackOff with k3s

Hi there,

I have a k3s cluster, where I want to use spegel. I install spegel as described in README using helm upgrade --create-namespace --namespace spegel --install --version v0.0.11 spegel oci://ghcr.io/xenitab/helm-charts/spegel

However, the pod gets into CrashLoopBackOff with the following error, when I check the logs:

Defaulted container "registry" out of: registry, configuration (init)
{"level":"error","ts":1695750332.7763488,"caller":"build/main.go:72","msg":"","error":"rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService","stacktrace":"main.main\n\t/build/main.go:72\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250"}

How can I solve this problem? I would appreciate any help.

Thanks a lot!

About this issue

Original URL
State: open
Created 9 months ago
Comments: 15 (4 by maintainers)

Most upvoted comments

Solution which worked:

Take content of /var/lib/rancher/k3s/agent/etc/containerd/config.toml
Create /var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl and add:

[plugins."io.containerd.grpc.v1.cri".registry]
  config_path = "/var/lib/rancher/k3s/agent/etc/containerd/certs.d"

Restart k3s-agent ( on worker nodes ) or k3s ( on master nodes )

lukaszraczylo on Nov 21, 2023

The error that you are seeing occurs because k3s uses a different path for its Containerd socket file. The default configuration uses a path that is common for most other flavors.

The issue with k3s is that support is a bit tricky currently until https://github.com/k3s-io/k3s/issues/5568 is fixed. There has been some push back against adopting the new Containerd configuration format for a while. It does however look like this feature will be implemented anyways. I was hoping that this would be fixed by now, the milestone has been moved a couple of times.

While it is in theory possible to make Spegel work with k3s, it is tricky and requires restarting Containerd on all nodes in the cluster. I do not think that most people are willing to do that. Instead I have been waiting for the issue to be solved until I spend time testing and documenting compatibility with k3s.

I will add a note about k3s to the compatibility docs, and update them when the issue is solved.

phillebaba on Sep 26, 2023

Looks like k3s uses a different path for its Containerd socket. Try changing the socket path and reinstall Spegel.

spegel:
  containerdSock: "/run/k3s/containerd/containerd.sock"

I will look at why it is printing the error message if the socket path is wrong.

phillebaba on Oct 5, 2023