ingress-nginx: Issue with install not working on cluster.

NGINX Ingress controller version: 0.41.2 via Helm chart 3.11.1

Kubernetes version (use kubectl version): 1.18.0 Server and 1.18.6+k3s1 Server

Raspberry pi 4 4GB cluster running the latest os.

What happened: I moved my cluster from my office to my basement and sometimes some pods do not turn back on properly I have never had a nginx issue until today though. So I used Helm and uninstalled it. Then tried to use helm to reinstall it however when I do this helm just hangs then reports this issue (Error: failed pre-install: timed out waiting for the condition).

Install method is as follows for clarity:

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo add stable https://charts.helm.sh/stable
helm repo update
helm install nginx-ingress ingress-nginx/ingress-nginx

Why is this happening? I can not find any information anywhere that would cause this. I have other pods that are not running correctly because this will not install. I Thought maybe that was the issue so I uninstalled those pods and still nothing. Can not seem to get it running.

Any and all help is appreciated.

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 3
Comments: 27 (11 by maintainers)

Most upvoted comments

So I found a work around

kubectl delete -A ValidatingWebhookConfiguration ingress-nginx-admission

but this doesnt seem like the best solution to me. is there something else?

11jwolfe2 on Nov 25, 2020

also experiencing what most here are. Deployment failed using https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.41.2/deploy/static/provider/cloud/deploy.yaml

time="2020-12-02T09:27:06+11:00" level=warning msg="Error retrieving artifact cache, not using skaffold cache: yaml: control characters are not allowed"
Tags used in deployment:
time="2020-12-02T09:27:08+11:00" level=warning msg="image [k8s.gcr.io/ingress-nginx/controller] is not used by the deployment"
 - namespace/ingress-nginx created
 - serviceaccount/ingress-nginx created
 - configmap/ingress-nginx-controller created
 - clusterrole.rbac.authorization.k8s.io/ingress-nginx created
 - clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
 - role.rbac.authorization.k8s.io/ingress-nginx created
 - service/ingress-nginx-controller-admission created
 - service/ingress-nginx-controller created
 - deployment.apps/ingress-nginx-controller created
 - validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
 - serviceaccount/ingress-nginx-admission created
 - clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
 - clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
 - role.rbac.authorization.k8s.io/ingress-nginx-admission created
 - rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
 - job.batch/ingress-nginx-admission-create created
 - job.batch/ingress-nginx-admission-patch created
Waiting for deployments to stabilize...
 - ingress-nginx:deployment/ingress-nginx-controller: creating container controller
    - ingress-nginx:pod/ingress-nginx-controller-869f588cf4-df9qd: creating container controller
 - ingress-nginx:deployment/ingress-nginx-controller: creating container controller
    - ingress-nginx:pod/ingress-nginx-controller-869f588cf4-df9qd: creating container controller
 - ingress-nginx:deployment/ingress-nginx-controller failed. Error: creating container controller.
1/1 deployment(s) failed

The logs and output of various failed pods

NAME                                        READY   STATUS              RESTARTS   AGE
ingress-nginx-admission-create-8f5w5        0/1     ImagePullBackOff    0          7m44s
ingress-nginx-admission-patch-lvcwv         0/1     ImagePullBackOff    0          7m44s
ingress-nginx-controller-869f588cf4-df9qd   0/1     ContainerCreating   0          7m44s
PS D:\source\gl\jofajard\k8-tools> kubectl get pods --namespace kube-system  
NAME                                     READY   STATUS             RESTARTS   AGE
coredns-777fb94669-psx9g                 0/1     Running            1          36m
coredns-777fb94669-qxrl8                 0/1     Running            1          36m
etcd-docker-desktop                      1/1     Running            1          33m
kube-apiserver-docker-desktop            1/1     Running            1          33m
kube-controller-manager-docker-desktop   1/1     Running            1          33m
kube-proxy-26wc7                         0/1     ImagePullBackOff   0          36m
kube-scheduler-docker-desktop            1/1     Running            1          33m
storage-provisioner                      0/1     CrashLoopBackOff   10         33m
vpnkit-controller                        1/1     Running            0          33m
PS D:\source\gl\jofajard\k8-tools> kubectl describe pod ingress-nginx-controller-869f588cf4-df9qd --namespace ingress-nginx
Name:           ingress-nginx-controller-869f588cf4-df9qd
Namespace:      ingress-nginx
Priority:       0
Node:           docker-desktop/192.168.65.3
Start Time:     Wed, 02 Dec 2020 09:27:10 +1100
Labels:         app.kubernetes.io/component=controller
                app.kubernetes.io/instance=ingress-nginx
                app.kubernetes.io/managed-by=skaffold
                app.kubernetes.io/name=ingress-nginx
                pod-template-hash=869f588cf4
                skaffold.dev/run-id=09598b4f-d070-40f1-bbf3-39a2aa296cc8
Annotations:    <none>
Status:         Pending
IP:
IPs:            <none>
Controlled By:  ReplicaSet/ingress-nginx-controller-869f588cf4
Containers:
  controller:
    Container ID:
    Image:         k8s.gcr.io/ingress-nginx/controller:v0.41.2@sha256:1f4f402b9c14f3ae92b11ada1dfe9893a88f0faeb0b2f4b903e2c67a0c3bf0de
    Image ID:
    Ports:         80/TCP, 443/TCP, 8443/TCP
    Host Ports:    0/TCP, 0/TCP, 0/TCP
    Args:
      /nginx-ingress-controller
      --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
      --election-id=ingress-controller-leader
      --ingress-class=nginx
      --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
      --validating-webhook=:8443
      --validating-webhook-certificate=/usr/local/certificates/cert
      --validating-webhook-key=/usr/local/certificates/key
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Requests:
      cpu:      100m
      memory:   90Mi
    Liveness:   http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=5
    Readiness:  http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=3
    Environment:
      POD_NAME:       ingress-nginx-controller-869f588cf4-df9qd (v1:metadata.name)
      POD_NAMESPACE:  ingress-nginx (v1:metadata.namespace)
      LD_PRELOAD:     /usr/local/lib/libmimalloc.so
    Mounts:
      /usr/local/certificates/ from webhook-cert (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from ingress-nginx-token-qsrvp (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  webhook-cert:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  ingress-nginx-admission
    Optional:    false
  ingress-nginx-token-qsrvp:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  ingress-nginx-token-qsrvp
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  kubernetes.io/os=linux
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason       Age                   From                     Message
  ----     ------       ----                  ----                     -------
  Normal   Scheduled    10m                   default-scheduler        Successfully assigned ingress-nginx/ingress-nginx-controller-869f588cf4-df9qd to docker-desktop
  Warning  FailedMount  2m4s (x4 over 8m53s)  kubelet, docker-desktop  Unable to attach or mount volumes: unmounted volumes=[webhook-cert], unattached volumes=[webhook-cert ingress-nginx-token-qsrvp]: timed out waiting for the condition
  Warning  FailedMount  40s (x13 over 10m)    kubelet, docker-desktop  MountVolume.SetUp failed for volume "webhook-cert" : secret "ingress-nginx-admission" not found```

liquidboy on Dec 1, 2020