kubernetes: kubectl delete should wait for resource to be deleted before returning

As per https://github.com/kubernetes/kubernetes/pull/40714#issuecomment-278460041, kubectl delete should ensure that the resource is deleted before returning by default. We can also add a --wait-for-deletion flag that users can set if they dont want to wait.

Work items:

  • First send DELETE request to apiserver with the resource name. Server will return back resource UID as part of the response.
  • Then keep sending DELETE requests to apiserver with UID precondition until server returns 404 or 409 or we timeout.

cc @liggitt @smarterclayton @bgrant0607 @kubernetes/sig-cli-bugs

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Reactions: 27
  • Comments: 27 (16 by maintainers)

Commits related to this issue

Most upvoted comments

Guys, it’s not a good idea to break current behaviour. By default delete command was not waiting until everything will be complete. I have a lot of scripts that drop a lot of resources. Now they are working for a very long time. Wait feature is good, but default behaviour change is upsetting

Yikes - was wondering why delete seems to now hang. Now I know! IMO a bad choice for the default behaviour 😦

Why was this made default? I cannot find any rationale for this here. IMO, one should have very good reason to break current behaviour. Besides, while I agree that this is a good addition, there are still plenty of use cases for not waiting.

@roffe sure, i found how to achieve this, thank you. My message was about breaking default behaviour.

This is driving me up the wall. I may get to this before too long.

What is actually going on under the covers? It appears that the command immediately prints “<object> deleted” and then hangs, but actually it is waiting. It seems like the messaging is incorrect. It should say “deleting” and if --wait is the default it should only print “deleted” when the object is actually deleted.

The way it works right now it seems like kubectl deletes things and then hangs for no reason, and it doesn’t give any indication that a state change has actually happened when the command finally terminates. (However ctrl-c and inspecting with kubectl get shows what is going on.) It’s also a little surprising that the whole operation is async and ctrl-c doesn’t stop it.)

I do think this is good default behavior it just needs better messaging.

The current state is that kubectl delete always waits (for ex: it waits for pods to be deleted when running kubectl delete rs or kubectl delete deployment). Namespace deletion and pod graceful deletion are exceptions.

@nikhiljindal I disagree with this statement for a couple reasons:

  1. replicasets and deployments are exceptions, not vice versa, because this behavior is driven by reapers which are optional (so by default kubectl does not wait - for example, Secret, ConfigMap and CRDs don’t have reapers)
  2. even for deployments the statement “kubectl delete always waits” is not 100% correct.

Example: I added a custom finalizer example.com/preventDeletion to deployment, and the behavior was the following:

  1. Created a deployment - all good:
❯ kubectl apply -f deploy.yaml
deployment "nginx-deployment" created

❯ kubectl get pod
NAME                                READY     STATUS    RESTARTS   AGE
nginx-deployment-6c54bd5869-m5tc9   1/1       Running   0          2s
nginx-deployment-6c54bd5869-t2pjl   1/1       Running   0          2s

❯ kubectl get deploy
NAME               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
nginx-deployment   2         2         2            2           11s
  1. Deleted a deployment - pods are terminated, but deployment is still there despite the message deployment "nginx-deployment" deleted, and it will remain there until I manually remove the example.com/preventDeletion finalizer.
❯ kubectl delete deploy nginx-deployment
deployment "nginx-deployment" deleted

❯ kubectl get pod
NAME                                READY     STATUS        RESTARTS   AGE
nginx-deployment-6c54bd5869-m5tc9   0/1       Terminating   0          29s
nginx-deployment-6c54bd5869-t2pjl   0/1       Terminating   0          29s

❯ kubectl get deploy
NAME               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
nginx-deployment   0         0         0            0           36s

e.g. 7 minutes later:

❯ kubectl get pod
No resources found.

❯ kubectl get deploy
NAME               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
nginx-deployment   0         0         0            0           7m