democratic-csi: Unable to mount volume after node reboot

I recently had to shutdown the entire k8s cluster. Prior to shutting it down, all pods/volumes etc were working correctly. However following the reboot, one of the deployments fails to start as it is unable to mount the pvc volume.

The error message is MountVolume.MountDevice failed for volume "pvc-3f761acf-262b-40ba-8b06-f76f205c70a4" : rpc error: code = Internal desc = {"code":1,"stdout":"Couldn't find valid filesystem superblock.\n","stderr":"resize2fs 1.44.5 (15-Dec-2018)\nresize2fs: Superblock checksum does not match superblock while trying to open /dev/sdb\n"}

As I wrote, all I did was take the cluster down, and then restart it. All other deployments have come back as expected. Including others using PVCs on the same NAS unit.

There are no obvious errors on the NAS unit.

Do you have any idea of the cause of this, and more importantly, given that I cannot start the deployment, how can I repair this?

Thanks

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 25 (13 by maintainers)

Commits related to this issue

Most upvoted comments

Got it! That did it. Thank you for taking the time to review and provide some help! Appreciate it.

Brilliant! Thank you!

Resolved in v1.3.0.

Enable the fs checks in your config as shown here: https://github.com/democratic-csi/democratic-csi/blob/master/examples/node-common.yaml