kubernetes: delete ingress object hangs
What happened:
delete ingress object hangs even with --force and --grace-period=0
$ k delete ingress ingress-2048 -n game-2048 --force --grace-period=0
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
ingress.extensions "ingress-2048" force deleted
.........
no events in game-2048 namespace
$ k get event -n game-2048
No resources found in game-2048 namespace.
What you expected to happen: deleted
How to reproduce it (as minimally and precisely as possible): it might be hard to reproduce
Anything else we need to know?:
on a eks cluster, the ingress is created from this doc, after delete aws load balancer controller, i can’t delete ingress object
Environment:
- Kubernetes version (use
kubectl version
):
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.3", GitCommit:"1e11e4a2108024935ecfcb2912226cedeafd99df", GitTreeState:"clean", BuildDate:"2020-10-14T18:49:28Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"17+", GitVersion:"v1.17.9-eks-a84824", GitCommit:"a84824a1c7b79853d0c57143ea42c9de1788dbba", GitTreeState:"clean", BuildDate:"2020-08-05T01:08:51Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
- Cloud provider or hardware configuration:
- OS (e.g:
cat /etc/os-release
): - Kernel (e.g.
uname -a
): - Install tools:
- Network plugin and version (if this is a network-related bug):
- Others:
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 5
- Comments: 21 (4 by maintainers)
yes, there is a finalizer, after read some docs and know what exactly finalizer is.
i remember i deleted the alb resource in aws management console before, so it always failed. after
kubectl edit
and remove finalizers, i can delete itso i’ll close it.
thanks for your help !
Use following command to patch and delete the ingress-2048
kubectl patch ingress $Ingressname -n $namespace -p ‘{“metadata”:{“finalizers”:[]}}’ --type=merge
The correct command is without stdout sign:
Does the load balancer add a finalizer to the ingress object? (you can check with
kubectl get ingress $name -o yaml
). If so, the object will not be removed until that finalizer is removed.You can make delete return without waiting by adding
-wait false
, but for the object to actually be deleted the finalizer must be removed.@jmmen1 try
It fails on the following error because of admission controller
Error from server (InternalError): Internal error occurred: failed calling webhook "vingress.elbv2.k8s.aws": the server could not find the requested resource
run:
kubectl delete ValidatingWebhookConfiguration aws-load-balancer-webhook
and then try to delete the ingress.
HI, I am I am facing the same issue. The ALB was deleted, the ingress object is still alive:
$ kubectl patch ingress argocd-server -n argocd -p '{"metadata":{"finalizers":[]}}' --type=merge
Here the output of the above command:
Error from server (InternalError): Internal error occurred: failed calling webhook "vingress.elbv2.k8s.aws": the server could not find the requested resource
The Logs:
{"level":"debug","ts":44444444,"logger":"secrets-manager","msg":"Monitoring secrets","groupID":"argocd/argocd-server","secrets":null} {"level":"error","ts":44444444,"logger":"controller-runtime.manager.controller.ingress","msg":"Reconciler error","name":"argocd-server","namespace":"argocd","error":"Internal error occurred: failed calling webhook \"vingress.elbv2.k8s.aws\": the server could not find the requested resource"}
Someone, can give any tips ?
tnx. This is life saver.
Hello, I faced the same issue, deleting the below sections from the ingress resource made me able to delete it. Make sure that aws-load-balancer-controller and aws-webhook installed successfully in the cluster
This worked for me too… I was struggling with this dangling ingress… Thanks…!!
thank u so much!!!
I have also ended up in this situation, using the aws-load-balancer-controller. Sadly, deleting and then recreating the controller still causes kubectl to hang when deleting the ingress.
Anyone else have more ideas for me to try?
update: I was able to delete the resource, by reinstalling alb-ingress-controller again and then my k8 controller was able to successfully remove the ingress object. then I removed the alb-ingress-controlled again. 😄
I used:
and then I removed the finalizers but still the ingress is not deleted. This is in AWS EKS.
Do I need to do anything else for the changes to be applied?