kubeadm: Image prepull fails with PodSecurityPolicy present

Is this a BUG REPORT or FEATURE REQUEST?

BUG REPORT

Versions

kubeadm version (use kubeadm version): v1.12.2

Environment:

• Kubernetes version (use kubectl version): v1.12.2
• Cloud provider or hardware configuration: VMware vSphere
• OS (e.g. from /etc/os-release): Debian 9 (Stretch)
• Kernel (e.g. uname -a): Linux 4.9.0-8-amd64 #1 SMP Debian 4.9.110-3+deb9u6 (2018-10-08)
• Others: PodSecurityPolicy is enabled and configured

What happened?

kubeadm upgrade apply v1.12.2 hangs on the ‘upgrade/prepull’ stage:

[upgrade/prepull] Will prepull images for components [kube-apiserver kube-controller-manager kube-scheduler etcd]
[upgrade/prepull] Prepulling image for component etcd.
[upgrade/prepull] Prepulling image for component kube-apiserver.
[upgrade/prepull] Prepulling image for component kube-controller-manager.
[upgrade/prepull] Prepulling image for component kube-scheduler.
[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-etcd
[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager
[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler
[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-apiserver

It turns out this is because the prepull pod doesn’t specify in its security context that it will be run as root, and thus gets assigned a restricted pod security policy that requires it to run as non-root. This gets rejected with an Error: container has runAsNonRoot and image will run as root and the pod will not start.

What you expected to happen?

Pod starts and continues without problems, even with a restricted PodSecurityPolicy. This can be achieved by specifying runAsNonRoot: false in the pod’s security context (which ensures it gets assigned a more privileged PodSecurityPolicy) or by specifying some non-root user that the pod should run as (which will allow it to run under the restricted PodSecurityPolicy).

How to reproduce it (as minimally and precisely as possible)?

Enable restricted pod security policy and execute kubeadm upgrade apply.

Anything else we need to know?

A workaround is to patch the prepull pods’ specs after they get registered by kubeadm:

for ds in $(kubectl -n kube-system get ds -o name | grep prepull | cut -d/ -f2); kubectl -n kube-system patch ds $ds  -p'{"spec":{"template":{"spec":{"securityContext":{"runAsNonRoot":false}}}}}'; end