rancher: Rancher doesn't work with arm32v7 (Raspberry Pi4 Cluster)
Hi, I just imported a k3s RasPi 4 cluster into Rancher and it’s stuck in pending state. It looks like it’s stuck “pending” because of an image pulling error.
kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
cattle-system cattle-cluster-agent-5d5c498cf-mbt28 0/1 ImagePullBackOff 0 32m
cattle-system cattle-node-agent-5w2dw 0/1 ImagePullBackOff 0 32m
cattle-system cattle-node-agent-j5wv6 0/1 ImagePullBackOff 0 32m
cattle-system cattle-node-agent-npt96 0/1 ImagePullBackOff 0 32m
cattle-system cattle-node-agent-sfqlb 0/1 ImagePullBackOff 0 32m
kube-system coredns-b7464766c-hqw8t 1/1 Running 2 48m
kube-system helm-install-traefik-lrh2z 0/1 CrashLoopBackOff 9 48m
kubectl get events --all-namespaces
NAMESPACE LAST SEEN TYPE REASON OBJECT MESSAGE
cattle-system 29m Normal Scheduled pod/cattle-cluster-agent-5d5c498cf-mbt28 Successfully assigned cattle-system/cattle-cluster-agent-5d5c498cf-mbt28 to node-3
cattle-system 28m Normal Pulling pod/cattle-cluster-agent-5d5c498cf-mbt28 Pulling image "rancher/rancher-agent:v2.3.0"
cattle-system 28m Warning Failed pod/cattle-cluster-agent-5d5c498cf-mbt28 Failed to pull image "rancher/rancher-agent:v2.3.0": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/rancher/rancher-agent:v2.3.0": fail
ed to unpack image on snapshotter overlayfs: no match for platform in manifest sha256:5933fb5d6669c790aa65d0937f92f48d8143ae60b5bd181ef1da3fa41e0fc4e4: not found
cattle-system 28m Warning Failed pod/cattle-cluster-agent-5d5c498cf-mbt28 Error: ErrImagePull
cattle-system 4m44s Normal BackOff pod/cattle-cluster-agent-5d5c498cf-mbt28 Back-off pulling image "rancher/rancher-agent:v2.3.0"
cattle-system 27m Warning Failed pod/cattle-cluster-agent-5d5c498cf-mbt28 Error: ImagePullBackOff
cattle-system 29m Normal SuccessfulCreate replicaset/cattle-cluster-agent-5d5c498cf Created pod: cattle-cluster-agent-5d5c498cf-mbt28
cattle-system 29m Normal ScalingReplicaSet deployment/cattle-cluster-agent Scaled up replica set cattle-cluster-agent-5d5c498cf to 1
cattle-system 29m Normal Scheduled pod/cattle-node-agent-5w2dw Successfully assigned cattle-system/cattle-node-agent-5w2dw to node-4
cattle-system 28m Normal Pulling pod/cattle-node-agent-5w2dw Pulling image "rancher/rancher-agent:v2.3.0"
cattle-system 28m Warning Failed pod/cattle-node-agent-5w2dw Failed to pull image "rancher/rancher-agent:v2.3.0": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/rancher/rancher-agent:v2.3.0": fail
ed to unpack image on snapshotter overlayfs: no match for platform in manifest sha256:5933fb5d6669c790aa65d0937f92f48d8143ae60b5bd181ef1da3fa41e0fc4e4: not found
cattle-system 28m Warning Failed pod/cattle-node-agent-5w2dw Error: ErrImagePull
cattle-system 4m37s Normal BackOff pod/cattle-node-agent-5w2dw Back-off pulling image "rancher/rancher-agent:v2.3.0"
cattle-system 14m Warning Failed pod/cattle-node-agent-5w2dw Error: ImagePullBackOff
cattle-system 29m Normal Scheduled pod/cattle-node-agent-j5wv6 Successfully assigned cattle-system/cattle-node-agent-j5wv6 to node-2
cattle-system 28m Normal Pulling pod/cattle-node-agent-j5wv6 Pulling image "rancher/rancher-agent:v2.3.0"
cattle-system 28m Warning Failed pod/cattle-node-agent-j5wv6 Failed to pull image "rancher/rancher-agent:v2.3.0": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/rancher/rancher-agent:v2.3.0": fail
ed to unpack image on snapshotter overlayfs: no match for platform in manifest sha256:5933fb5d6669c790aa65d0937f92f48d8143ae60b5bd181ef1da3fa41e0fc4e4: not found
cattle-system 28m Warning Failed pod/cattle-node-agent-j5wv6 Error: ErrImagePull
cattle-system 4m32s Normal BackOff pod/cattle-node-agent-j5wv6 Back-off pulling image "rancher/rancher-agent:v2.3.0"
cattle-system 24m Warning Failed pod/cattle-node-agent-j5wv6 Error: ImagePullBackOff
cattle-system 29m Normal Scheduled pod/cattle-node-agent-npt96 Successfully assigned cattle-system/cattle-node-agent-npt96 to node-1
cattle-system 28m Normal Pulling pod/cattle-node-agent-npt96 Pulling image "rancher/rancher-agent:v2.3.0"
Does the image not exists?
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 18
- Comments: 44 (3 by maintainers)
As listed the issue is you are not running the 64bit version of Raspbian. I’m running three PI4’s with SSD only.
Steps I followed to get my cluster up:
Adjust Raspbian to run 64bit, adding this line informs rpi-update to update with the 64bit version.
Update: /boot/config.txt with arm_64bit=1
Run rpi-update to update the pi to 64bit
To confirm run uname -a
(Source: https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=250730)
1 - Setup Master run the following to install K3S: curl -sfL https://get.k3s.io | sh -s - --write-kubeconfig-mode 644
Validate that it’s up: sudo kubectl get nodes sudo systemctl status k3s.service
Make sure the cluster is up and happy before adding the workers.
2 - Setup the Worker On the master export the token from: sudo cat /var/lib/rancher/k3s/server/node-token
On the worker: export K3S_TOKEN=“paste above token here” export K3S_URL=https://<your master ip>:6443 curl -sfL https://get.k3s.io | sh -
Then create the cluster in Rancher, Import custom.
Run the two commands to add the user and add the master that Rancher gives you. kubectl create clusterrolebinding cluster-admin-binding --clusterrole cluster-admin --user pi sudo kubectl apply -f https://<rancher server ip>/v3/import/2r3131231231231231231231231313123.yaml
Reopening this issue because we recently had someone in the Rancher Users Slack asking for help importing an ARM cluster as well.
I’m also seeing this issue on a RPi 4 with the latest Rasperry Pi OS with bullseye. Pointing to the image tage
v2.6.2-linux-arm64allows me to download the image, however, it will not run on k3s due to an incompatible architecture:Found this blog:
Since I already have a rancher server, I have to import an rpi3 k3s cluster. With above instruction, I can deploy the rancher-agent:v2.5.3-linux-arm64 successfully on my raspberry pi 3/B+ k3s cluster to import it into my Rancher server.
More detailed instructions: >>
1. Prepare (Optional)
2. K3s server on a raspberry pi 3 b+
3. Import k3s cluster into your Rancher server.
4. Add some workers
That’s all folks.
Good luck!
Please find a solution. We love rancher!
To append onto what @SolaceZA and @felts94 suggested, I was able to use my existing Rancher 2.4.5 to adopt a current 1.16 K3S cluster.
I was able to convert existing pi’s to arm64 using @SolaceZA suggested steps. This process seems to have worked nicely without damaging anything I was paying attention to.
to @felts94 point I did find(though I am not sure if this is a sequencing problem or an actual issue) that I needed to edit BOTH the
deployment/cattle-cluster-agentanddaemonset/cattle-node-agentand explicitly force the use of *-linux-arm64 images. From here the cluster seems okay and I can confirm that I have been able to add nodes repeatedly using this method.Ok this worked,
I also changed the deployment file to the image name rancher/rancher-agent:v2.5.1-linux-arm64 in this step
curl --insecure -sfL https:// /v3/import/dlzwxczz5vcrx4xk5kq4t97s8l2bm6zwf2cptx946txjjsbtxhv4kk.yaml | kubectl apply -f -
k3os and rpi-update might be the magic. But please check if rpi-update is a must.
Current version of k3os arm64 images does NOT support rpi 3 b+. See https://github.com/rancher/k3os/blob/master/README.md#arm-overlay-installation.
And of course, you can prepare or remaster the SD card. I’ll be glad to see your successful. 😉
最后咋解决的啊
yes, just rancher/rancher-agent no rancher-agent-armhf image only linux-amd64,linux-arm64,windows-amd64
ubuntu arm64 K3S docker :No container started