minikube: verifying ingress addon pods : timed out waiting for the condition: timed out waiting for the condition

Because the image is not pulled, try workaround:

$ kubectl get pod -A

List all the system pod names

$ kubectl describe pod/ingress-nginx-controller-xxxxxx --namespace=kube-system

Find out the image name which has not been pulled successfully, in my case is quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.33.0

$ minikube ssh
$ docker pull quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.33.0

Pull the image in minikube vm manully

I faced the same I found the problem from my poor internet connection

So, for me, edititng the deployment with the correct image URL seems to have worked.

To do this, just: kubectl -n kube-system edit deployment ingress-nginx-controller and look for “image:” and change it to:

image: us.gcr.io/k8s-artifacts-prod/ingress-nginx/controller@sha256:0e072dddd1f7f8fc8909a2ca6f65e76c5f0d2fcfb8be47935ae3457e8bbceb20

I ran to this issue on minikube version v1.26.0 too. steps that I’ve done to solve it:

• type kubectl get pods -n ingress-nginx and you should see something like the following:

NAME                                        READY   STATUS      RESTARTS        AGE
ingress-nginx-admission-create-zhp94        0/1     Completed   0               3h59m
ingress-nginx-admission-patch-ggrd8         0/1     Completed   0               3h59m
ingress-nginx-controller-755dfbfc65-xtrs5   0/1     ImagePullBackOff     1 (3h11m ago)   3h59m

• and then if you do kubectl describe pod -n ingress-nginx <pod-name> you should be able to see all of the logs within the pod. I checked the logs of ingress-nginx-controller-755dfbfc65-xtrs5 and saw its cant download the image.

• I logged in to minikube container using minikube ssh

• then manually download ingress-nginx-controller image using below command:

time docker pull k8s.gcr.io/ingress-nginx/controller:v1.2.1@sha256:5516d103a9c2ecc4f026efbd4b40662ce22dc1f824fb129ed121460aaa5c47f8

Note: I needed to download v1.2.1 of ingress-nginx but if you need to download a newer or older version you can check the google containers registry to find correct sha256:xxxx

Still no solution for this problem ??? I’ve been struggling for a while for my project and I have to support soon thank you in advance for any solution that is what i have

🔎 Verifying ingress addon…

💣 enable failed: run callbacks: running callbacks: [verifying ingress addon pods : timed out waiting for the condition: timed out waiting for the condition]

😿 minikube is exiting due to an error. If the above message is not useful, open an issue: 👉 https://github.com/kubernetes/minikube/issues/new/choose

Thanks @renich ! Met same issue. After kubectl -n kube-system edit deployment ingress-nginx-controller, change

us.gcr.io/k8s-artifacts-prod/ingress-nginx/controller:v0.34.1@sha256:0e072dddd1f7f8fc8909a2ca6f65e76c5f0d2fcfb8be47935ae3457e8bbceb20

to

us.gcr.io/k8s-artifacts-prod/ingress-nginx/controller@sha256:0e072dddd1f7f8fc8909a2ca6f65e76c5f0d2fcfb8be47935ae3457e8bbceb20

(You can try verify by opening first URL in browser, which gets 404 error, the second one which is also version 0.34.1 which can be opened)

Basically just need remove :v0.34.1.

After editing, sine it is Vim, just type :wq or :x and hit Enter.

And then do minikube addons enable ingress again.

Having something so fundamentally broken like ingress is fairly disheartening…

minikube addons enable ingress --images=“KubeWebhookCertgenCreate=registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1,KubeWebhookCertgenPatch=registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1,IngressController=google_containers/nginx-ingress-controller:v1.2.1” --registries=“IngressController=registry.cn-hangzhou.aliyuncs.com”

So, for me, edititng the deployment with the correct image URL seems to have worked.

To do this, just: kubectl -n kube-system edit deployment ingress-nginx-controller and look for “image:” and change it to:

image: us.gcr.io/k8s-artifacts-prod/ingress-nginx/controller@sha256:0e072dddd1f7f8fc8909a2ca6f65e76c5f0d2fcfb8be47935ae3457e8bbceb20

WOOWWW Thank You!! its work!

In my case I had something running on port 80, that can also happen.

For reference, here’s my full minikube start attempt:

[wnilges@thingsboard-minikube root]$  minikube start --addons=ingress --alsologtostderr -v=5

(It’s really big)

It seems that the URL being used isn’t correct. If you go here: https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/US/ingress-nginx/controller@sha256:0e072dddd1f7f8fc8909a2ca6f65e76c5f0d2fcfb8be47935ae3457e8bbceb20 it works, but the URL logged is: us.gcr.io/k8s-artifacts-prod/ingress-nginx/controller:v0.34.1@sha256:0e072dddd1f7f8fc8909a2ca6f65e76c5f0d2fcfb8be47935ae3457e8bbceb20

If you put them side by side, ìt’s easy to notice the difference:

us.gcr.io/k8s-artifacts-prod/ingress-nginx/controller@sha256:0e072dddd1f7f8fc8909a2ca6f65e76c5f0d2fcfb8be47935ae3457e8bbceb20
us.gcr.io/k8s-artifacts-prod/ingress-nginx/controller:v0.34.1@sha256:0e072dddd1f7f8fc8909a2ca6f65e76c5f0d2fcfb8be47935ae3457e8bbceb20

Same problem. I followed your steps but my problem was: Warning FailedMount 12m (x7 over 12m) kubelet, minikube MountVolume.SetUp failed for volume “webhook-cert” : secret “ingress-nginx-admission” not found

So I realized it was SELinux. If you have SELinux enforcing, try using permissive mode.

Worked for me!

Because the image is not pulled, try workaround:

$ kubectl get pod -A

List all the system pod names

$ kubectl describe pod/ingress-nginx-controller-xxxxxx --namespace=kube-system

Find out the image name which has not been pulled successfully, in my case is quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.33.0

$ minikube ssh
$ docker pull quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.33.0

Pull the image in minikube vm manully

I did some searching and found this- looks like the registry used for the IngressController does not support using a tag and image digest in the same URL the way that docker.io does. https://github.com/google/go-containerregistry/issues/351

Maybe I’m mistaken and GCR doesn’t work exactly like this (the issue I linked was closed with a merge request quite some time ago) but it does seem to be behaving similarly. This page offers two distinct solutions for pulling from GCR: https://cloud.google.com/container-registry/docs/pushing-and-pulling#pulling_images_from_a_registry

Thanks @renich ! Met same issue. After kubectl -n kube-system edit deployment ingress-nginx-controller, change

us.gcr.io/k8s-artifacts-prod/ingress-nginx/controller:v0.34.1@sha256:0e072dddd1f7f8fc8909a2ca6f65e76c5f0d2fcfb8be47935ae3457e8bbceb20

to

us.gcr.io/k8s-artifacts-prod/ingress-nginx/controller@sha256:0e072dddd1f7f8fc8909a2ca6f65e76c5f0d2fcfb8be47935ae3457e8bbceb20

(You can try verify by opening first URL in browser, which gets 404 error, the second one which is also version 0.34.1 which can be opened)

Basically just need remove :v0.34.1.

After editing, sine it is Vim, just type :wq or :x and hit Enter.

And then do minikube addons enable ingress again.

Just add another update. In later installation, this bug is still there, however, it seems not affecting anything. Even I have a new fresh installation from ground. What I did is just

minikube start
minikube addons enable ingress

and let minikube retry at the background, eventually it will succeed. The longest time takes like 3 hours after over 10 times retry I think.

I faced the same I found the problem from my poor internet connection did u get it fix later on ?

@tstromberg update: after some more tests, I see that most of the times it gives the error, but sometimes not. As you said, the max waiting time may be on the edge of the average time it takes. I would like to isolate which version update is causing this increased time so that we can upstream this issue to the component with degraded performance.

How can I do that? Do I need to recompile minikube from source or there is an easier way to do it?