operator-sdk: CR with finalizer hang when the namespace is deleted because of Ansible operator is allowing the deletion of the operator before the deletion of the CR be accomplished.

Bug Report

CR with finalizer hang when the namespace is deleted because of Ansible operator is allowing the deletion of the operator before the deletion of the CR to be accomplished.

NOTE: Issue opened in order to make clear the scenario/bug raised in #1493

What did you do?

---
- version: v1alpha1
  group: cache.example.com
  kind: Memcached
  role: /opt/ansible/roles/memcached
  finalizer:
    name: finalizer.cache.example.com
  • Create an namespace and apply the example.
$ oc new-project memcached

$ kubectl create -f deploy/crds/cache_v1alpha1_memcached_crd.yaml
$ kubectl create -f deploy/service_account.yaml
$ kubectl create -f deploy/role.yaml
$ kubectl create -f deploy/role_binding.yaml
$ kubectl create -f deploy/operator.yaml
$ kubectl create -f deploy/crds/cache_v1alpha1_memcached_cr.yaml
  • Try to delete the namespace
$ oc project delete memcached

What did you expect to see? The CR + Operator + Namespace be deleted with success.

What did you see instead? Under which circumstances? The namespace is marked to be deleted, the operator is deleted, but the CR is not which not allows the namespace to be deleted as well and is hanging it.

  • Reason: The operator has been deleted before the CR then it cannot remove the finalizer metadata from it which causes the bug.

  • Workarround: manual deletion of the finalizer metadata from the CR which would be made by the operator if it was not deleted first. E.g oc patch memcached example-memcached -p '{"metadata":{"finalizers": []}}' --type=merge OR

Delete the CR before deleting the namespace for the operator be able to remove the finalizer metadata. oc delete deploy/crds/cache_v1alpha1_memcached_cr.yaml

Environment

  • operator-sdk version: 0.8.1
  • go version: go version go1.12.5 darwin/amd64
  • Kubernetes version information:
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.4", GitCommit:"c27b913fddd1a6c480c229191a087698aa92f0b1", GitTreeState:"clean", BuildDate:"2019-03-01T23:34:27Z", GoVersion:"go1.12", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"11+", GitVersion:"v1.11.0+d4cacc0", GitCommit:"d4cacc0", GitTreeState:"clean", BuildDate:"2019-05-02T11:52:09Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}
  • Kubernetes cluster kind: Minishift
  • Are you writing your operator in ansible, helm, or go? Ansible

Additional context Following the images to illustrate the bug.

  • Namespaced marked to be deleted and hanged by the CR with finalizer.
Screenshot 2019-05-31 at 10 22 53
  • See that the operator was deleted before the CR. Screenshot 2019-05-31 at 10 23 27

  • See that the CR still there with the finalizer metadata which should be removed by the operator. Screenshot 2019-05-31 at 10 23 20

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 15 (7 by maintainers)

Most upvoted comments

My original replication test code/procedures for this issue can also be found here, in case it is helpful: https://github.com/operator-framework/operator-sdk/issues/1493#issuecomment-497485434