cri-o: cri-o 1.17.0 breaks ARM clusters
After updating crio to 1.17.0 my arm cluster got broken, the essential pause image could no longer be pulled, pods do not start after the upgrade since the images are cross-build on amd64 for ARM64 resulting in an image that is in fact usable on arm64 but cri-o refuses to try.
Trying to pull manually:
$ crictl pull k8s.gcr.io/pause-arm64:3.1
FATA[0001] pulling image failed: rpc error: code = Unknown desc = Image architecture mismatch: image uses "amd64", expecting "arm64"
From the logfiles
Feb 08 17:10:16 n2-master0 kubelet[6717]: E0208 17:10:16.050473 6717 kuberuntime_sandbox.go:68] CreatePodSandbox for pod "kube-apiserver-n2-master0_kube-system(c0554e8450d1ed33e8b37ef2bd7d6b60)" failed: rpc error: code = Unknown desc = error creating pod sandbox with name "k8s_kube-apiserver-n2-master0_kube-system_c0554e8450d1ed33e8b37ef2bd7d6b60_0": Image architecture mismatch: image uses "amd64", expecting "arm64"
The general problem was reported in this issue https://github.com/containers/libpod/issues/4849 a workaround for podman was implemented.
As long as crossplatform builds result in images tagged with the wrong architecture I see it as a bug in cri-o to be unable to use the images that worked in the past. afaik there is no easy fix for the bad tagging, especially with images build on dockerhub with limited options to influence the build process.
Current workaround: Downgrade to cri-o 1.16.1
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 38 (17 by maintainers)
sorry but is it right that builds for Raspbian failed, see: https://build.opensuse.org/package/live_build_log/devel:kubic:libcontainers:stable:cri-o:1.17/cri-o/Raspbian_10/armv7l
it also seems that there isn’t a new build at: https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.17/Raspbian_10/
btw. it seems that I can’t reopen the issue
@therevoman AFAICT, this will be fixed in https://github.com/cri-o/cri-o/pull/3797. I will work on backporting the required PRs to 1.17 and 1.16. thanks for the patience folks 👍