minikube: Exiting due to MK_ADDON_ENABLE: run callbacks: running callbacks: [waiting for app.kubernetes.io/name=ingress-nginx pods: timed out waiting for the condition]

What Happened?

   ~  minikube addons enable ingress --alsologtostderr

I0323 16:01:01.216663 1306 out.go:298] Setting OutFile to fd 1 … I0323 16:01:01.216790 1306 out.go:350] isatty.IsTerminal(1) = true I0323 16:01:01.216795 1306 out.go:311] Setting ErrFile to fd 2… I0323 16:01:01.216799 1306 out.go:350] isatty.IsTerminal(2) = true I0323 16:01:01.216876 1306 root.go:313] Updating PATH: /Users/anikmazumder/.minikube/bin I0323 16:01:01.217257 1306 config.go:177] Loaded profile config “freeway”: Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.22.2 I0323 16:01:01.217271 1306 addons.go:65] Setting ingress=true in profile “freeway” I0323 16:01:01.217278 1306 addons.go:153] Setting addon ingress=true in “freeway” I0323 16:01:01.217499 1306 host.go:66] Checking if “freeway” exists … I0323 16:01:01.217825 1306 main.go:130] libmachine: Found binary path at /Users/anikmazumder/.minikube/bin/docker-machine-driver-hyperkit I0323 16:01:01.217859 1306 main.go:130] libmachine: Launching plugin server for driver hyperkit I0323 16:01:01.224666 1306 main.go:130] libmachine: Plugin server listening at address 127.0.0.1:49813 I0323 16:01:01.225032 1306 main.go:130] libmachine: () Calling .GetVersion I0323 16:01:01.225437 1306 main.go:130] libmachine: Using API Version 1 I0323 16:01:01.225449 1306 main.go:130] libmachine: () Calling .SetConfigRaw I0323 16:01:01.225703 1306 main.go:130] libmachine: () Calling .GetMachineName I0323 16:01:01.226031 1306 main.go:130] libmachine: Found binary path at /Users/anikmazumder/.minikube/bin/docker-machine-driver-hyperkit I0323 16:01:01.226057 1306 main.go:130] libmachine: Launching plugin server for driver hyperkit I0323 16:01:01.232575 1306 main.go:130] libmachine: Plugin server listening at address 127.0.0.1:49817 I0323 16:01:01.232895 1306 main.go:130] libmachine: () Calling .GetVersion I0323 16:01:01.233213 1306 main.go:130] libmachine: Using API Version 1 I0323 16:01:01.233225 1306 main.go:130] libmachine: () Calling .SetConfigRaw I0323 16:01:01.233454 1306 main.go:130] libmachine: () Calling .GetMachineName I0323 16:01:01.233547 1306 main.go:130] libmachine: (freeway) Calling .GetState I0323 16:01:01.233661 1306 main.go:130] libmachine: (freeway) DBG | exe=/Users/anikmazumder/.minikube/bin/docker-machine-driver-hyperkit uid=0 I0323 16:01:01.233749 1306 main.go:130] libmachine: (freeway) DBG | hyperkit pid from json: 1064 I0323 16:01:01.234553 1306 main.go:130] libmachine: (freeway) Calling .DriverName I0323 16:01:01.237028 1306 out.go:177] ▪ Using image k8s.gcr.io/ingress-nginx/controller:v1.0.0-beta.3 ▪ Using image k8s.gcr.io/ingress-nginx/controller:v1.0.0-beta.3 I0323 16:01:01.241242 1306 out.go:177] ▪ Using image k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0 ▪ Using image k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0 I0323 16:01:01.245228 1306 out.go:177] ▪ Using image k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0 ▪ Using image k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0 I0323 16:01:01.245412 1306 addons.go:337] installing /etc/kubernetes/addons/ingress-deploy.yaml I0323 16:01:01.245429 1306 ssh_runner.go:319] scp memory --> /etc/kubernetes/addons/ingress-deploy.yaml (17265 bytes) I0323 16:01:01.245454 1306 main.go:130] libmachine: (freeway) Calling .GetSSHHostname I0323 16:01:01.245607 1306 main.go:130] libmachine: (freeway) Calling .GetSSHPort I0323 16:01:01.245703 1306 main.go:130] libmachine: (freeway) Calling .GetSSHKeyPath I0323 16:01:01.245780 1306 main.go:130] libmachine: (freeway) Calling .GetSSHUsername I0323 16:01:01.245863 1306 sshutil.go:53] new ssh client: &{IP:192.168.64.40 Port:22 SSHKeyPath:/Users/anikmazumder/.minikube/machines/freeway/id_rsa Username:docker} I0323 16:01:01.286135 1306 ssh_runner.go:152] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.22.2/kubectl apply -f /etc/kubernetes/addons/ingress-deploy.yaml I0323 16:01:01.483582 1306 main.go:130] libmachine: Making call to close driver server I0323 16:01:01.483606 1306 main.go:130] libmachine: (freeway) Calling .Close I0323 16:01:01.483787 1306 main.go:130] libmachine: Successfully made call to close driver server I0323 16:01:01.483800 1306 main.go:130] libmachine: Making call to close connection to plugin binary I0323 16:01:01.483836 1306 main.go:130] libmachine: (freeway) DBG | Closing plugin on server side I0323 16:01:01.483850 1306 main.go:130] libmachine: Making call to close driver server I0323 16:01:01.483859 1306 main.go:130] libmachine: (freeway) Calling .Close I0323 16:01:01.483969 1306 main.go:130] libmachine: Successfully made call to close driver server I0323 16:01:01.483981 1306 main.go:130] libmachine: Making call to close connection to plugin binary I0323 16:01:01.483996 1306 addons.go:375] Verifying addon ingress=true in “freeway” I0323 16:01:01.484013 1306 main.go:130] libmachine: (freeway) DBG | Closing plugin on server side I0323 16:01:01.487472 1306 out.go:177] 🔎 Verifying ingress addon… 🔎 Verifying ingress addon… I0323 16:01:01.491406 1306 kapi.go:75] Waiting for pod with label “app.kubernetes.io/name=ingress-nginx” in ns “ingress-nginx” … I0323 16:01:01.501779 1306 kapi.go:86] Found 3 Pods for label selector app.kubernetes.io/name=ingress-nginx I0323 16:01:01.501795 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:01:02.008043 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:01:02.504787 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:01:03.005432 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:01:03.504870 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:01:04.008896 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:01:04.508135 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:01:05.005302 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:01:05.508197 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:01:06.006344 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:01:06.505529 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:01:07.005438 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:01:07.506910 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:01:08.004505 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:01:22.005397 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:01:22.505472 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>]

I0323 16:01:53.508420 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:01:54.008542 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:01:54.509345 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>]

I0323 16:07:00.511729 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:07:01.014840 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:07:01.509177 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:07:01.511157 1306 kapi.go:96] waiting for pod “app.kubernetes.io/name=ingress-nginx”, current state: Pending: [<nil>] I0323 16:07:01.511168 1306 kapi.go:108] duration metric: took 6m0.016870726s to wait for app.kubernetes.io/name=ingress-nginx … I0323 16:07:01.514432 1306 out.go:177]

W0323 16:07:01.514494 1306 out.go:242] ❌ Exiting due to MK_ADDON_ENABLE: run callbacks: running callbacks: [waiting for app.kubernetes.io/name=ingress-nginx pods: timed out waiting for the condition] ❌ Exiting due to MK_ADDON_ENABLE: run callbacks: running callbacks: [waiting for app.kubernetes.io/name=ingress-nginx pods: timed out waiting for the condition] W0323 16:07:01.514502 1306 out.go:242]

W0323 16:07:01.516149 1306 out.go:242] ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ │ │ 😿 If the above advice does not help, please let us know: │ │ 👉 https://github.com/kubernetes/minikube/issues/new/choose │ │ │ │ Please run minikube logs --file=logs.txt and attach logs.txt to the GitHub issue. │ │ Please also attach the following file to the GitHub issue: │ │ - /var/folders/tq/kzxpxzrn2jv_mhhq7x9zxyz80000gp/T/minikube_addons_40d123381e5225c70d5a73cffc85fbb84b01a7be_0.log │ │ │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ │ │ 😿 If the above advice does not help, please let us know: │ │ 👉 https://github.com/kubernetes/minikube/issues/new/choose │ │ │ │ Please run minikube logs --file=logs.txt and attach logs.txt to the GitHub issue. │ │ Please also attach the following file to the GitHub issue: │ │ - /var/folders/tq/kzxpxzrn2jv_mhhq7x9zxyz80000gp/T/minikube_addons_40d123381e5225c70d5a73cffc85fbb84b01a7be_0.log │ │ │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ I0323 16:07:01.518640 1306 out.go:177]

Attach the log file

certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""" pod=“ingress-nginx/ingress-nginx-admission-create–1-wzhdg” podUID=2ead59ff-2de3-4e1b-aad2-bcd4c9cb7426 Mar 23 10:12:45 freeway kubelet[2696]: E0323 10:12:45.989077 2696 remote_image.go:114] “PullImage from image service failed” err=“rpc error: code = Unknown desc = Error response from daemon: Get "https://k8s.gcr.io/v2/\”: dial tcp 142.251.10.82:443: i/o timeout" image=“k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068” Mar 23 10:12:45 freeway kubelet[2696]: E0323 10:12:45.989549 2696 kuberuntime_image.go:51] “Failed to pull image” err=“rpc error: code = Unknown desc = Error response from daemon: Get "https://k8s.gcr.io/v2/\”: dial tcp 142.251.10.82:443: i/o timeout" image=“k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068” Mar 23 10:12:45 freeway kubelet[2696]: E0323 10:12:45.989666 2696 kuberuntime_manager.go:895] container &Container{Name:patch,Image:k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068,Command:[],Args:[patch --webhook-name=ingress-nginx-admission --namespace=$(POD_NAMESPACE) --patch-mutating=false --secret-name=ingress-nginx-admission --patch-failure-policy=Fail],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:POD_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-xd7pw,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,} start failed in pod ingress-nginx-admission-patch–1-8kfxf_ingress-nginx(c1d4899a-85c8-4a6a-a7fa-c5fa055542b0): ErrImagePull: rpc error: code = Unknown desc = Error response from daemon: Get “https://k8s.gcr.io/v2/”: dial tcp 142.251.10.82:443: i/o timeout Mar 23 10:12:45 freeway kubelet[2696]: E0323 10:12:45.989754 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "patch" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://k8s.gcr.io/v2/\\\”: dial tcp 142.251.10.82:443: i/o timeout"" pod=“ingress-nginx/ingress-nginx-admission-patch–1-8kfxf” podUID=c1d4899a-85c8-4a6a-a7fa-c5fa055542b0 Mar 23 10:12:55 freeway kubelet[2696]: E0323 10:12:55.982872 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "create" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-create–1-wzhdg” podUID=2ead59ff-2de3-4e1b-aad2-bcd4c9cb7426 Mar 23 10:13:00 freeway kubelet[2696]: E0323 10:13:00.983217 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "patch" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-patch–1-8kfxf” podUID=c1d4899a-85c8-4a6a-a7fa-c5fa055542b0 Mar 23 10:13:06 freeway kubelet[2696]: E0323 10:13:06.983507 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "create" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-create–1-wzhdg” podUID=2ead59ff-2de3-4e1b-aad2-bcd4c9cb7426 Mar 23 10:13:08 freeway kubelet[2696]: E0323 10:13:08.043525 2696 remote_image.go:114] “PullImage from image service failed” err=“rpc error: code = Unknown desc = Error response from daemon: Get "https://gcr.io/v2/\”: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" image=“gcr.io/k8s-minikube/minikube-ingress-dns:0.0.1@sha256:69dc3c878c2e49ad85b70fdf9e8e6e87a1f961f42c8029e0912bebfa828ade46” Mar 23 10:13:08 freeway kubelet[2696]: E0323 10:13:08.043568 2696 kuberuntime_image.go:51] “Failed to pull image” err=“rpc error: code = Unknown desc = Error response from daemon: Get "https://gcr.io/v2/\”: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" image=“gcr.io/k8s-minikube/minikube-ingress-dns:0.0.1@sha256:69dc3c878c2e49ad85b70fdf9e8e6e87a1f961f42c8029e0912bebfa828ade46” Mar 23 10:13:08 freeway kubelet[2696]: E0323 10:13:08.043625 2696 kuberuntime_manager.go:895] container &Container{Name:minikube-ingress-dns,Image:gcr.io/k8s-minikube/minikube-ingress-dns:0.0.1@sha256:69dc3c878c2e49ad85b70fdf9e8e6e87a1f961f42c8029e0912bebfa828ade46,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:,HostPort:53,ContainerPort:53,Protocol:UDP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:DNS_PORT,Value:53,ValueFrom:nil,},EnvVar{Name:POD_IP,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:status.podIP,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-hp7mh,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:nil,Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,} start failed in pod kube-ingress-dns-minikube_kube-system(ce10d907-ff21-47ea-b632-ac97bd299700): ErrImagePull: rpc error: code = Unknown desc = Error response from daemon: Get “https://gcr.io/v2/”: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Mar 23 10:13:08 freeway kubelet[2696]: E0323 10:13:08.043652 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "minikube-ingress-dns" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://gcr.io/v2/\\\”: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"" pod=“kube-system/kube-ingress-dns-minikube” podUID=ce10d907-ff21-47ea-b632-ac97bd299700 Mar 23 10:13:08 freeway kubelet[2696]: E0323 10:13:08.382107 2696 secret.go:195] Couldn’t get secret ingress-nginx/ingress-nginx-admission: secret “ingress-nginx-admission” not found Mar 23 10:13:08 freeway kubelet[2696]: E0323 10:13:08.382203 2696 nestedpendingoperations.go:301] Operation for “{volumeName:kubernetes.io/secret/6ad45ba2-995a-42b4-b851-3672f52139a2-webhook-cert podName:6ad45ba2-995a-42b4-b851-3672f52139a2 nodeName:}” failed. No retries permitted until 2022-03-23 10:15:10.382188301 +0000 UTC m=+1598.908288224 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume “webhook-cert” (UniqueName: “kubernetes.io/secret/6ad45ba2-995a-42b4-b851-3672f52139a2-webhook-cert”) pod “ingress-nginx-controller-69bdbc4d57-dqhdx” (UID: “6ad45ba2-995a-42b4-b851-3672f52139a2”) : secret “ingress-nginx-admission” not found Mar 23 10:13:13 freeway kubelet[2696]: E0323 10:13:13.983326 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "patch" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-patch–1-8kfxf” podUID=c1d4899a-85c8-4a6a-a7fa-c5fa055542b0 Mar 23 10:13:15 freeway kubelet[2696]: E0323 10:13:15.981813 2696 kubelet.go:1720] “Unable to attach or mount volumes for pod; skipping pod” err=“unmounted volumes=[webhook-cert], unattached volumes=[webhook-cert kube-api-access-ftsrz]: timed out waiting for the condition” pod=“ingress-nginx/ingress-nginx-controller-69bdbc4d57-dqhdx” Mar 23 10:13:15 freeway kubelet[2696]: E0323 10:13:15.982454 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“unmounted volumes=[webhook-cert], unattached volumes=[webhook-cert kube-api-access-ftsrz]: timed out waiting for the condition” pod=“ingress-nginx/ingress-nginx-controller-69bdbc4d57-dqhdx” podUID=6ad45ba2-995a-42b4-b851-3672f52139a2 Mar 23 10:13:19 freeway kubelet[2696]: E0323 10:13:19.984093 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "create" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-create–1-wzhdg” podUID=2ead59ff-2de3-4e1b-aad2-bcd4c9cb7426 Mar 23 10:13:19 freeway kubelet[2696]: E0323 10:13:19.985010 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "minikube-ingress-dns" with ImagePullBackOff: "Back-off pulling image \"gcr.io/k8s-minikube/minikube-ingress-dns:0.0.1@sha256:69dc3c878c2e49ad85b70fdf9e8e6e87a1f961f42c8029e0912bebfa828ade46\""” pod=“kube-system/kube-ingress-dns-minikube” podUID=ce10d907-ff21-47ea-b632-ac97bd299700 Mar 23 10:13:28 freeway kubelet[2696]: E0323 10:13:28.983556 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "patch" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-patch–1-8kfxf” podUID=c1d4899a-85c8-4a6a-a7fa-c5fa055542b0 Mar 23 10:13:31 freeway kubelet[2696]: E0323 10:13:31.983717 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "create" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-create–1-wzhdg” podUID=2ead59ff-2de3-4e1b-aad2-bcd4c9cb7426 Mar 23 10:13:32 freeway kubelet[2696]: E0323 10:13:32.982641 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "minikube-ingress-dns" with ImagePullBackOff: "Back-off pulling image \"gcr.io/k8s-minikube/minikube-ingress-dns:0.0.1@sha256:69dc3c878c2e49ad85b70fdf9e8e6e87a1f961f42c8029e0912bebfa828ade46\""” pod=“kube-system/kube-ingress-dns-minikube” podUID=ce10d907-ff21-47ea-b632-ac97bd299700 Mar 23 10:13:42 freeway kubelet[2696]: E0323 10:13:42.983463 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "patch" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-patch–1-8kfxf” podUID=c1d4899a-85c8-4a6a-a7fa-c5fa055542b0 Mar 23 10:13:46 freeway kubelet[2696]: E0323 10:13:46.983825 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "minikube-ingress-dns" with ImagePullBackOff: "Back-off pulling image \"gcr.io/k8s-minikube/minikube-ingress-dns:0.0.1@sha256:69dc3c878c2e49ad85b70fdf9e8e6e87a1f961f42c8029e0912bebfa828ade46\""” pod=“kube-system/kube-ingress-dns-minikube” podUID=ce10d907-ff21-47ea-b632-ac97bd299700 Mar 23 10:13:46 freeway kubelet[2696]: E0323 10:13:46.983826 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "create" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-create–1-wzhdg” podUID=2ead59ff-2de3-4e1b-aad2-bcd4c9cb7426 Mar 23 10:13:57 freeway kubelet[2696]: E0323 10:13:57.983286 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "create" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-create–1-wzhdg” podUID=2ead59ff-2de3-4e1b-aad2-bcd4c9cb7426 Mar 23 10:13:57 freeway kubelet[2696]: E0323 10:13:57.988510 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "patch" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-patch–1-8kfxf” podUID=c1d4899a-85c8-4a6a-a7fa-c5fa055542b0 Mar 23 10:13:59 freeway kubelet[2696]: E0323 10:13:59.983601 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "minikube-ingress-dns" with ImagePullBackOff: "Back-off pulling image \"gcr.io/k8s-minikube/minikube-ingress-dns:0.0.1@sha256:69dc3c878c2e49ad85b70fdf9e8e6e87a1f961f42c8029e0912bebfa828ade46\""” pod=“kube-system/kube-ingress-dns-minikube” podUID=ce10d907-ff21-47ea-b632-ac97bd299700 Mar 23 10:14:09 freeway kubelet[2696]: E0323 10:14:09.984073 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "create" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-create–1-wzhdg” podUID=2ead59ff-2de3-4e1b-aad2-bcd4c9cb7426 Mar 23 10:14:10 freeway kubelet[2696]: E0323 10:14:10.983281 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "minikube-ingress-dns" with ImagePullBackOff: "Back-off pulling image \"gcr.io/k8s-minikube/minikube-ingress-dns:0.0.1@sha256:69dc3c878c2e49ad85b70fdf9e8e6e87a1f961f42c8029e0912bebfa828ade46\""” pod=“kube-system/kube-ingress-dns-minikube” podUID=ce10d907-ff21-47ea-b632-ac97bd299700 Mar 23 10:14:12 freeway kubelet[2696]: E0323 10:14:12.983454 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "patch" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-patch–1-8kfxf” podUID=c1d4899a-85c8-4a6a-a7fa-c5fa055542b0 Mar 23 10:14:22 freeway kubelet[2696]: E0323 10:14:22.982934 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "minikube-ingress-dns" with ImagePullBackOff: "Back-off pulling image \"gcr.io/k8s-minikube/minikube-ingress-dns:0.0.1@sha256:69dc3c878c2e49ad85b70fdf9e8e6e87a1f961f42c8029e0912bebfa828ade46\""” pod=“kube-system/kube-ingress-dns-minikube” podUID=ce10d907-ff21-47ea-b632-ac97bd299700 Mar 23 10:14:22 freeway kubelet[2696]: E0323 10:14:22.982934 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "create" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-create–1-wzhdg” podUID=2ead59ff-2de3-4e1b-aad2-bcd4c9cb7426 Mar 23 10:14:24 freeway kubelet[2696]: E0323 10:14:24.983195 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "patch" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-patch–1-8kfxf” podUID=c1d4899a-85c8-4a6a-a7fa-c5fa055542b0 Mar 23 10:14:34 freeway kubelet[2696]: E0323 10:14:34.983916 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "minikube-ingress-dns" with ImagePullBackOff: "Back-off pulling image \"gcr.io/k8s-minikube/minikube-ingress-dns:0.0.1@sha256:69dc3c878c2e49ad85b70fdf9e8e6e87a1f961f42c8029e0912bebfa828ade46\""” pod=“kube-system/kube-ingress-dns-minikube” podUID=ce10d907-ff21-47ea-b632-ac97bd299700 Mar 23 10:14:35 freeway kubelet[2696]: E0323 10:14:35.983175 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "create" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-create–1-wzhdg” podUID=2ead59ff-2de3-4e1b-aad2-bcd4c9cb7426 Mar 23 10:14:37 freeway kubelet[2696]: E0323 10:14:37.982864 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "patch" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-patch–1-8kfxf” podUID=c1d4899a-85c8-4a6a-a7fa-c5fa055542b0 Mar 23 10:14:46 freeway kubelet[2696]: E0323 10:14:46.984368 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "minikube-ingress-dns" with ImagePullBackOff: "Back-off pulling image \"gcr.io/k8s-minikube/minikube-ingress-dns:0.0.1@sha256:69dc3c878c2e49ad85b70fdf9e8e6e87a1f961f42c8029e0912bebfa828ade46\""” pod=“kube-system/kube-ingress-dns-minikube” podUID=ce10d907-ff21-47ea-b632-ac97bd299700 Mar 23 10:14:46 freeway kubelet[2696]: E0323 10:14:46.984408 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "create" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-create–1-wzhdg” podUID=2ead59ff-2de3-4e1b-aad2-bcd4c9cb7426 Mar 23 10:14:51 freeway kubelet[2696]: E0323 10:14:51.983102 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "patch" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-patch–1-8kfxf” podUID=c1d4899a-85c8-4a6a-a7fa-c5fa055542b0 Mar 23 10:14:58 freeway kubelet[2696]: E0323 10:14:58.983622 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "create" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-create–1-wzhdg” podUID=2ead59ff-2de3-4e1b-aad2-bcd4c9cb7426 Mar 23 10:15:00 freeway kubelet[2696]: E0323 10:15:00.982705 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "minikube-ingress-dns" with ImagePullBackOff: "Back-off pulling image \"gcr.io/k8s-minikube/minikube-ingress-dns:0.0.1@sha256:69dc3c878c2e49ad85b70fdf9e8e6e87a1f961f42c8029e0912bebfa828ade46\""” pod=“kube-system/kube-ingress-dns-minikube” podUID=ce10d907-ff21-47ea-b632-ac97bd299700 Mar 23 10:15:04 freeway kubelet[2696]: E0323 10:15:04.983587 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "patch" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-patch–1-8kfxf” podUID=c1d4899a-85c8-4a6a-a7fa-c5fa055542b0 Mar 23 10:15:10 freeway kubelet[2696]: E0323 10:15:10.422476 2696 secret.go:195] Couldn’t get secret ingress-nginx/ingress-nginx-admission: secret “ingress-nginx-admission” not found Mar 23 10:15:10 freeway kubelet[2696]: E0323 10:15:10.422554 2696 nestedpendingoperations.go:301] Operation for “{volumeName:kubernetes.io/secret/6ad45ba2-995a-42b4-b851-3672f52139a2-webhook-cert podName:6ad45ba2-995a-42b4-b851-3672f52139a2 nodeName:}” failed. No retries permitted until 2022-03-23 10:17:12.422539614 +0000 UTC m=+1720.948639536 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume “webhook-cert” (UniqueName: “kubernetes.io/secret/6ad45ba2-995a-42b4-b851-3672f52139a2-webhook-cert”) pod “ingress-nginx-controller-69bdbc4d57-dqhdx” (UID: “6ad45ba2-995a-42b4-b851-3672f52139a2”) : secret “ingress-nginx-admission” not found Mar 23 10:15:13 freeway kubelet[2696]: E0323 10:15:13.983354 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "create" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-create–1-wzhdg” podUID=2ead59ff-2de3-4e1b-aad2-bcd4c9cb7426 Mar 23 10:15:14 freeway kubelet[2696]: E0323 10:15:14.982810 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "minikube-ingress-dns" with ImagePullBackOff: "Back-off pulling image \"gcr.io/k8s-minikube/minikube-ingress-dns:0.0.1@sha256:69dc3c878c2e49ad85b70fdf9e8e6e87a1f961f42c8029e0912bebfa828ade46\""” pod=“kube-system/kube-ingress-dns-minikube” podUID=ce10d907-ff21-47ea-b632-ac97bd299700 Mar 23 10:15:18 freeway kubelet[2696]: E0323 10:15:18.983162 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "patch" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-patch–1-8kfxf” podUID=c1d4899a-85c8-4a6a-a7fa-c5fa055542b0 Mar 23 10:15:24 freeway kubelet[2696]: E0323 10:15:24.983230 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "create" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-create–1-wzhdg” podUID=2ead59ff-2de3-4e1b-aad2-bcd4c9cb7426 Mar 23 10:15:29 freeway kubelet[2696]: E0323 10:15:29.983087 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "minikube-ingress-dns" with ImagePullBackOff: "Back-off pulling image \"gcr.io/k8s-minikube/minikube-ingress-dns:0.0.1@sha256:69dc3c878c2e49ad85b70fdf9e8e6e87a1f961f42c8029e0912bebfa828ade46\""” pod=“kube-system/kube-ingress-dns-minikube” podUID=ce10d907-ff21-47ea-b632-ac97bd299700 Mar 23 10:15:31 freeway kubelet[2696]: E0323 10:15:31.983385 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "patch" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-patch–1-8kfxf” podUID=c1d4899a-85c8-4a6a-a7fa-c5fa055542b0 Mar 23 10:15:33 freeway kubelet[2696]: E0323 10:15:33.982658 2696 kubelet.go:1720] “Unable to attach or mount volumes for pod; skipping pod” err=“unmounted volumes=[webhook-cert], unattached volumes=[kube-api-access-ftsrz webhook-cert]: timed out waiting for the condition” pod=“ingress-nginx/ingress-nginx-controller-69bdbc4d57-dqhdx” Mar 23 10:15:33 freeway kubelet[2696]: E0323 10:15:33.982703 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“unmounted volumes=[webhook-cert], unattached volumes=[kube-api-access-ftsrz webhook-cert]: timed out waiting for the condition” pod=“ingress-nginx/ingress-nginx-controller-69bdbc4d57-dqhdx” podUID=6ad45ba2-995a-42b4-b851-3672f52139a2 Mar 23 10:15:35 freeway kubelet[2696]: E0323 10:15:35.983892 2696 pod_workers.go:765] “Error syncing pod, skipping” err=“failed to "StartContainer" for "create" with ImagePullBackOff: "Back-off pulling image \"k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068\""” pod=“ingress-nginx/ingress-nginx-admission-create–1-wzhdg” podUID=2ead59ff-2de3-4e1b-aad2-bcd4c9cb7426

  • ==> kubernetes-dashboard [52b16e89b40e] <==

  • 2022/03/23 09:49:25 Using namespace: kubernetes-dashboard 2022/03/23 09:49:25 Using in-cluster config to connect to apiserver 2022/03/23 09:49:25 Using secret token for csrf signing 2022/03/23 09:49:25 Initializing csrf token from kubernetes-dashboard-csrf secret 2022/03/23 09:49:25 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf 2022/03/23 09:49:25 Successful initial request to the apiserver, version: v1.22.2 2022/03/23 09:49:25 Generating JWE encryption key 2022/03/23 09:49:25 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting 2022/03/23 09:49:25 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard 2022/03/23 09:49:25 Initializing JWE encryption key from synchronized object 2022/03/23 09:49:25 Creating in-cluster Sidecar client 2022/03/23 09:49:25 Successful request to sidecar 2022/03/23 09:49:25 Serving insecurely on HTTP port: 9090 2022/03/23 09:49:25 Starting overwatch

  • ==> kubernetes-dashboard [f80a16a5f1b5] <==

  • 2022/03/23 09:48:40 Starting overwatch panic: Get “https://10.96.0.1:443/api/v1/namespaces/kubernetes-dashboard/secrets/kubernetes-dashboard-csrf”: dial tcp 10.96.0.1:443: i/o timeout

goroutine 1 [running]: github.com/kubernetes/dashboard/src/app/backend/client/csrf.(*csrfTokenManager).init(0xc0003c3ea0) /home/runner/work/dashboard/dashboard/src/app/backend/client/csrf/manager.go:41 +0x413 github.com/kubernetes/dashboard/src/app/backend/client/csrf.NewCsrfTokenManager(…) /home/runner/work/dashboard/dashboard/src/app/backend/client/csrf/manager.go:66 github.com/kubernetes/dashboard/src/app/backend/client.(*clientManager).initCSRFKey(0xc00049f480) /home/runner/work/dashboard/dashboard/src/app/backend/client/manager.go:502 +0xc6 github.com/kubernetes/dashboard/src/app/backend/client.(*clientManager).init(0xc00049f480) /home/runner/work/dashboard/dashboard/src/app/backend/client/manager.go:470 +0x47 github.com/kubernetes/dashboard/src/app/backend/client.NewClientManager(…) /home/runner/work/dashboard/dashboard/src/app/backend/client/manager.go:551 main.main() /home/runner/work/dashboard/dashboard/src/app/backend/dashboard.go:95 +0x21c 2022/03/23 09:48:40 Using namespace: kubernetes-dashboard 2022/03/23 09:48:40 Using in-cluster config to connect to apiserver 2022/03/23 09:48:40 Using secret token for csrf signing 2022/03/23 09:48:40 Initializing csrf token from kubernetes-dashboard-csrf secret

  • ==> storage-provisioner [20f37bb1cbda] <==

  • I0323 09:49:27.166601 1 storage_provisioner.go:116] Initializing the minikube storage provisioner… I0323 09:49:27.185118 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service! I0323 09:49:27.185416 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath… I0323 09:49:44.602882 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath I0323 09:49:44.603249 1 event.go:282] Event(v1.ObjectReference{Kind:“Endpoints”, Namespace:“kube-system”, Name:“k8s.io-minikube-hostpath”, UID:“53768986-6d46-4a1c-97f2-8ea30e27b469”, APIVersion:“v1”, ResourceVersion:“1276”, FieldPath:“”}): type: ‘Normal’ reason: ‘LeaderElection’ freeway_bf9213f4-febc-42c7-b991-19def3aa5f6a became leader I0323 09:49:44.603450 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_freeway_bf9213f4-febc-42c7-b991-19def3aa5f6a! I0323 09:49:44.705327 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_freeway_bf9213f4-febc-42c7-b991-19def3aa5f6a!

  • ==> storage-provisioner [6cc141a3cfb9] <==

  • I0323 09:48:40.818397 1 storage_provisioner.go:116] Initializing the minikube storage provisioner… F0323 09:49:10.831423 1 main.go:39] error getting server version: Get “https://10.96.0.1:443/version?timeout=32s”: dial tcp 10.96.0.1:443: i/o timeout

Operating System

No response

Driver

No response

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 25 (8 by maintainers)

Most upvoted comments

This happened to me today too. I ran; minikube addons enable ingress minikube addons enable ingress-dns minikube addons enable ingress

First ‘enable’ didn’t work, with the titled error. Second (line 3) ‘enable’, worked properly.

This happened to me today too. I ran; minikube addons enable ingress minikube addons enable ingress-dns minikube addons enable ingress

First ‘enable’ didn’t work, with the titled error. Second (line 3) ‘enable’, worked properly.

This happened to me today too. I ran; minikube addons enable ingress minikube addons enable ingress-dns minikube addons enable ingress

First ‘enable’ didn’t work, with the titled error. Second (line 3) ‘enable’, worked properly.

This work well. First we need to enable ingress-dns addon , then enable ingress

I’m seeing this when using v1.27.0 on macOS v12.6 (M1 Pro).

I have exactly same situation. In my case the ingress-controller pod does not start because since version v1.25.2 of minikube it has a toleration NoSchedule for node-role.kubernetes.io/master which is obviously set on the master node of minikube. Also there is a quite strange thing. When I do minikube addons disable ingress, it shows correct version of controller image:

$ minikube addons disable ingress
    ▪ Using image ingress-nginx/controller:v1.1.1
    ▪ Using image ingress-nginx/kube-webhook-certgen:v1.1.1
    ▪ Using image ingress-nginx/kube-webhook-certgen:v1.1.1
🌑  "The 'ingress' addon is disabled

But when I enable it back, there is another (old version)

$ minikube addons enable ingress
    ▪ Using image k8s.gcr.io/ingress-nginx/controller:v1.0.4
    ▪ Using image k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1
    ▪ Using image k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1
🔎  Verifying ingress addon...
🌟  The 'ingress' addon is enabled

And after that everything is just fine. I remember there was a mechanism to remove that toleration from ingress deployment for single node setup in old versions of minikube. Should it still work?

Hi @Orygeunik, this is definitely a different error than the initial post.

MK_ADDON_ENABLE: run callbacks: running callbacks: [waiting for app.kubernetes.io/name=ingress-nginx pods: timed out waiting for the condition] is a very generic error, it would be best if you opened a new issue specific to the problem you’re experiencing.