kubernetes: imagePullSecrets cannot work for the private infra_image

What happened: the infra_image is set to: harbor-local.unicloudsrv.com/muye/pause-amd64:3.1 and the image on my harbor

$ kubectl -n kube-system create secret docker-registry \
  harbor-local --docker-server=harbor-local.unicloudsrv.com  --docker-username=xxx --docker-password=xxx
$ kubectl create secret docker-registry \
  harbor-local --docker-server=harbor-local.unicloudsrv.com  --docker-username=xxx --docker-password=xxx
$ kubectl -n kube-system patch serviceaccount \
  default -p '{"imagePullSecrets": [{"name": "harbor-local"}]}'
$ kubectl patch serviceaccount \
  default -p '{"imagePullSecrets": [{"name": "harbor-local"}]}'

If there is no pause images on the node,kubelet will log

pod_workers.go:191] Error syncing pod 5436b18d-b391-4e01-89aa-eb3570df531a ("coredns-54796ff55-84d4n_kube-system(5436b18d-b391-4e01-89aa-eb3570df531a)"), skipping: failed to "CreatePodSandbox" for "coredns-54796ff55-84d4n_kube-system(5436b18d-b391-4e01-89aa-eb3570df531a)" with CreatePodSandboxError: "CreatePodSandbox for pod \"coredns-54796ff55-84d4n_kube-system(5436b18d-b391-4e01-89aa-eb3570df531a)\" failed: rpc error: code = Unknown desc = failed pulling image \"harbor-local.unicloudsrv.com/muye/pause-amd64:3.1\": Error response from daemon: pull access denied for harbor-local.unicloudsrv.com/muye/pause-amd64, repository does not exist or may require 'docker login': denied: requested access to the resource is denied"

but the info of the imagePullSecrets is ok, if I use the docker pull could pull the image. and the pod will be running after I use docker pull harbor-local.unicloudsrv.com/muye/pause-amd64:3.1

What you expected to happen:

I hope it can pull normally by kubelet

How to reproduce it (as minimally and precisely as possible): 1.There is no infra_image on the node 2.There is no pod on the all namespaces 3.The infra_image is on a Private Registry 4.This happens when you deploy pods in non-default ns, example I deploy the coredns in the kube-system

Environment:

# kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.7", GitCommit:"be3d344ed06bff7a4fc60656200a93c74f31f9a4", GitTreeState:"clean", BuildDate:"2020-02-11T19:34:02Z", GoVersion:"go1.13.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.7", GitCommit:"be3d344ed06bff7a4fc60656200a93c74f31f9a4", GitTreeState:"clean", BuildDate:"2020-02-11T19:24:46Z", GoVersion:"go1.13.6", Compiler:"gc", Platform:"linux/amd64"}

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 16 (4 by maintainers)

Most upvoted comments

kubelet can configure pause image for docker-shim only that will be deprecated later. The pause image is decided by cri in general, for instance, containerd can specify it in its configuration. I thinks it works as expected.

This is not a k8s cluster-scope issue. It is something that container runtime should concern about.

  • As Ted said, docker login to the specific repo for pause image pulling if the container runtime is docker.

see the command

kubectl -n kube-system patch serviceaccount default -p '{"imagePullSecrets": [{"name": "harbor-local"}]}'
kubectl patch serviceaccount default -p '{"imagePullSecrets": [{"name": "harbor-local"}]}'

This command is effective for image pull of pod, but not for infra_image

may require 'docker login': denied: requested access to the resource is denied

When you used docker pull, you would have logged in, hence command line worked.