kubernetes: [Failing Test] Cos containerd tests are failing due libgc mismatch
Which jobs are failing?
- https://testgrid.k8s.io/sig-node-containerd#cos-cgroupv1-containerd-e2e
- https://testgrid.k8s.io/sig-node-containerd#cos-cgroupv1-inplace-pod-resize-containerd-e2e-serial
- https://testgrid.k8s.io/sig-node-containerd#cos-cgroupv2-containerd-e2e
- https://testgrid.k8s.io/sig-node-presubmits#pr-kubelet-gce-cluster-e2e-inplace-pod-resize-containerd-main-v2
- https://testgrid.k8s.io/sig-node-containerd#e2e-cos-device-plugin-gpu
- ~~https://testgrid.k8s.io/sig-node-containerd#cos-cgroupv2-containerd-node-e2e~~
Which tests are failing?
These tests are completely failing due to a compilation issue so no tests are running.
Since when has it been failing?
NA
Testgrid link
- https://testgrid.k8s.io/sig-node-containerd#cos-cgroupv1-containerd-e2e
- https://testgrid.k8s.io/sig-node-containerd#cos-cgroupv1-inplace-pod-resize-containerd-e2e-serial
- https://testgrid.k8s.io/sig-node-containerd#cos-cgroupv2-containerd-e2e
- https://testgrid.k8s.io/sig-node-presubmits#pr-kubelet-gce-cluster-e2e-inplace-pod-resize-containerd-main-v2
- https://testgrid.k8s.io/sig-node-containerd#e2e-cos-device-plugin-gpu
- ~~https://testgrid.k8s.io/sig-node-containerd#cos-cgroupv2-containerd-node-e2e~~
Reason for failure (if possible)
Oct 17 18:13:43.987386 bootstrap-e2e-master containerd[1205]: /home/containerd/usr/local/bin/containerd: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /home/containerd/usr/local/bin/containerd)
Oct 17 18:13:43.784506 bootstrap-e2e-master systemd[1]: containerd.service: Main process exited, code=exited, status=1/FAILURE
Oct 17 18:13:43.784516 bootstrap-e2e-master systemd[1]: containerd.service: Failed with result 'exit-code'.
Oct 17 18:13:48.976159 bootstrap-e2e-master systemd[1]: containerd.service: Scheduled restart job, restart counter is at 1.
Oct 17 18:13:48.976351 bootstrap-e2e-master systemd[1]: Stopped containerd container runtime.
Oct 17 18:13:48.979390 bootstrap-e2e-master systemd[1]: Starting containerd container runtime...
Oct 17 18:13:48.981424 bootstrap-e2e-master systemd[1]: Started containerd container runtime.
Oct 17 18:13:48.986207 bootstrap-e2e-master containerd[1322]: /home/containerd/usr/local/bin/containerd: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /home/containerd/usr/local/bin/containerd)
You can go to the containerd logs in artifacts and see that GLIBC_2.34 is not found.
Anything else we need to know?
Typically these fixes can be resolved by an image update.
These tests don’t seem to point to any default image.
Relevant SIG(s)
/sig node
About this issue
- Original URL
- State: closed
- Created 8 months ago
- Comments: 16 (16 by maintainers)
Thank you.
ci-cos-cgroupv1-inplace-pod-resize-containerd-main-e2e-gce-serial is not triggered yet. Next run will be tomorrow. I hope it’s fixed.
Correct. However, we can still statically build containerd binaries or ask containerd maintainers to publish prebuilt static binaries from their main branch. With dynamically built executables we’re going to have this kind of issues in future.
/assign
I noticed that the jobs were passing with image
cos-stable-109-17800-0-47, started failing with imagecos-stable-109-17800-0-51. I’ve filed an internal support case, will post more updates later.