csi-digitalocean: Deleted node is not able to attach to existing PVC once is created again

What did you do? (required. The issue will be closed when not provided.)

Deleted a random node in my Rancher cluster to see how Percona Xtradb cluster behaved

What did you expect to happen?

Node to be recreated and Percona node attached to existing PVC

Configuration (MUST fill this out):

• system logs:

Please provide the following logs:

kubectl cluster-info dump > kubernetes-dump.log

This will output everthing from your cluster. Please use a private gist via https://gist.github.com/ to share this dump with us Not able to create a gist, is generating an error on the site but happy to send to an email if needed.

• manifests, such as pvc, deployments, etc… you used to reproduce: Deployed Percona using OperatorHub.io cr.zip

Please provide the total set of manifests that are needed to reproduce the issue. Just providing the pvc is not helpful. If you cannot provide it due privacy concerns, please try creating a reproducible case.

• CSI Version: https://github.com/digitalocean/csi-digitalocean/tree/master/deploy/kubernetes/releases/csi-digitalocean-latest

• Kubernetes Version: 1.18.3

• Cloud provider/framework version, if applicable (such as Rancher): RancherOS 2.4.5 -> DigitalOcean -> 3 nodes Not using DOKS.

  Normal Scheduled 76s default-scheduler Successfully assigned my-percona-xtradb-cluster-operator/cluster-01-pxc-2 to worker-pool2 Normal SuccessfulAttachVolume 76s attachdetach-controller AttachVolume.Attach succeeded for volume “pvc-e801f45f-3ac1-4d5e-8ce5-dc2a79191992” Warning FailedMount 28s (x7 over 60s) kubelet, worker-pool2 MountVolume.MountDevice failed for volume “pvc-e801f45f-3ac1-4d5e-8ce5-dc2a79191992” : rpc error: code = Internal desc = formatting disk failed: exit status 1 cmd: ‘mkfs.ext4 -F /dev/disk/by-id/scsi-0DO_Volume_pvc-e801f45f-3ac1-4d5e-8ce5-dc2a79191992’ output: “mke2fs 1.45.5 (07-Jan-2020)\nThe file /dev/disk/by-id/scsi-0DO_Volume_pvc-e801f45f-3ac1-4d5e-8ce5-dc2a79191992 does not exist and no size was specified.\n”

Hi, to reproduce create a Percona operator than a CR with 3 nodes, after cluster is running delete a node manually and wait for recreation. Volume will not bind correctly.

If I manually attach the volume in DO dashboard and terminate the pod the new pod gets created correctly.

Any help is appreciated.