amazon-vpc-cni-k8s: Coredns stuck on ContainerCreating with `FailedCreatePodSandBox` warning for CNI versions 1.7.6 with Cilium 1.9.1
What happened: New cluster with nodes restarted. coredns stuck on ContainerCreating when using CNI v1.7.6 and Cilium 1.9.1. Other pods are also experiencing the same behavior ( ContainerCreating )
coredns:v1.6.6-eksbuild.1
Attach logs
Warning FailedCreatePodSandBox 29s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "112861d5995ca8f44c1dc17f00c947d72a44cf69c9deda34fbaf56b204742874" network for pod "coredns-6d857998c6-gxsd7": networkPlugin cni failed to set up pod "coredns-6d857998c6-gxsd7_kube-system" network: invalid character '{' after top-level value
What you expected to happen: I expected coredns and other pods to be in running state
How to reproduce it (as minimally and precisely as possible): Deploy cni version 1.7.6 and cilium 1.9.1 on EKS 1.17
Anything else we need to know?: We have Cilium running in chaining mode (v1.9.1) [(https://docs.cilium.io/en/v1.9/gettingstarted/cni-chaining-aws-cni/)]
Environment:
- Kubernetes version (use
kubectl version
):
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.3", GitCommit:"1e11e4a2108024935ecfcb2912226cedeafd99df", GitTreeState:"clean", BuildDate:"2020-10-14T12:50:19Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"17+", GitVersion:"v1.17.12-eks-7684af", GitCommit:"7684af4ac41370dd109ac13817023cb8063e3d45", GitTreeState:"clean", BuildDate:"2020-10-20T22:57:40Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
- CNI Version
amazon-k8s-cni-init:v1.7.6
amazon-k8s-cni:v1.7.6
- OS (e.g:
cat /etc/os-release
):
NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"
- Kernel (e.g.
uname -a
): Linux REDACTED.compute.internal 4.14.203-156.332.amzn2.x86_64 #1 SMP Fri Oct 30 19:19:33 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 1
- Comments: 28 (12 by maintainers)
Hi @shaikatz
Sorry for the delay. Will take this up in rel 1.7.9 planned for January.
@jayanthvn what is the ETA to release that fix that doesn’t require any manual modification of the nodes? 2 versions already released after this PR was merged, but this fix was ignored in both of them.
Thanks again @jayanthvn