kubernetes: 1.10 kubelet doesn't start any daemonset containers with 1.12 control plane

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:

/kind bug

/kind feature

What happened:

I upgraded a kubernetes cluster with an existing 1.10 node to 1.12. From there on the kubelet didn’t start any containers. Pods got scheduled and then went into the MatchNodeSelector status

What you expected to happen:#

The kubelet to start containers

How to reproduce it (as minimally and precisely as possible):

Create a kubeadm 1.12 cluster, join a node with a 1.10 kubelet

Anything else we need to know?:

Bisecting revealed the patch that fixes this in the kubelet is b1a7f85c9e90ef84c51cd173a16e42ce2560fe35 which got introduced in #62453

cc @k82cn as you wrote the fixing patch

Environment:

  • Kubernetes version (use kubectl version):
  • Cloud provider or hardware configuration:
  • OS (e.g. from /etc/os-release):
  • Kernel (e.g. uname -a):
  • Install tools:
  • Others:

/sig scheduling

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 31 (31 by maintainers)

Most upvoted comments

prefer to give an known issues for that 😃

I don’t think that’s sufficient. That’s essentially a known issue of “kubelets are not actually supported at n-2 skew levels”. We should add a known issue to inform people considering upgrading their control plane to 1.12, but also take action to restore the claimed support for skew levels.

one possibility is to use the kubelet version reported by the node (status.nodeInfo.kubeletVersion) to determine whether to use a direct node assignment or matchfield affinity

I’m going to create a PR for 1.12 to handle that if no other options; 1.13 is not necessary.