longhorn: [BUG] StorageClass of pv and pvc of a recovered pv should not always be default.

Describe the bug

  • I follow this guide to create an encrypted PV: https://longhorn.io/docs/1.2.3/advanced-resources/volume-encryption/
  • created a StorageClass longhorn-crypto-global
  • then I created a PVC (in namespace “xyz”) that uses this storage class (longhorn-crypto-global) to get an encrypted volume
  • then I create a Deployment (in namespace “xyz”) that uses the pvc to create a pv
  • then I create a backup to S3 using Longhorn UI
  • then I delete the namespace “xyz”
  • then I restore the backup
  • then I create a pvc of the restored pv and check the “encrypt” checkbox
  • but looking at the pv and pvc this is the output:
k get pv -A
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                    STORAGECLASS             REASON   AGE
pvc-250db57b-8c2a-4288-b75a-aae34cc241e8   128Mi      RWO            Delete           Bound    traefik/traefik          longhorn-crypto-global            27h
pvc-2f164f3a-3089-475a-addb-dbb59a9e3fd0   100Mi      RWO            Retain           Bound    db/postgres-pvc          longhorn-static                   7s
pvc-f44fc526-c2e8-499b-a6a1-d67363a33f89   5Gi        RWO            Delete           Bound    monitor/storage-loki-0   longhorn-crypto-global            11h

k get pvc -A
NAMESPACE   NAME             STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS             AGE
xyz          postgres-pvc     Bound    pvc-2f164f3a-3089-475a-addb-dbb59a9e3fd0   100Mi      RWO            longhorn-static          12s
monitor     storage-loki-0   Bound    pvc-f44fc526-c2e8-499b-a6a1-d67363a33f89   5Gi        RWO            longhorn-crypto-global   11h
traefik     traefik          Bound    pvc-250db57b-8c2a-4288-b75a-aae34cc241e8   128Mi      RWO            longhorn-crypto-global   27h

  • you see that the recovered pv and pvc has the storage class longhorn-static but both should have longhorn-crypto-global.

My guess is that it uses the default storage class. EDIT: I checked that. If you use the UI to change the default storage class to longhorn-crypto-global then it is set when the pv is recovered from backup. But as I said above - that info should not be taken from the default setting but stored in the backup and taken from there. The problem is that the storage class of the pvc can not be changed (edited) with kubectl edit.

The StorageClass should be stored with the backup nd restored for pv and pvc.

Environment

  • Longhorn version: 1.2.3
  • Installation method (e.g. Rancher Catalog App/Helm/Kubectl): helm
  • Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version: kubeadm
    • Number of management node in the cluster: 1
    • Number of worker node in the cluster: none
  • Node config
    • OS type and version: Debian 11.01
    • CPU per node: 8 hyperthreads
    • Memory per node: 32 GB
    • Disk type(e.g. SSD/NVMe): mechanical
    • Network bandwidth between the nodes: none
  • Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal): Baremetal
  • Number of Longhorn volumes in the cluster: 3

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 7
  • Comments: 18 (8 by maintainers)

Most upvoted comments

Longhorn uses the default Longhorn Static StorageClass Name longhorn-static when PV/PVC is created.

And yes, we could improve this one with solutions:

* During backup, record the storage class name. And during restore, use the storage class name found in the backup metadata. It's good for a bunch of backups to restore at one time.

* During restoring and creating PV/PVC, let the user choose which StorageClass name to use. This provides more granularity to choose different storage class names per volume.

Or do both. During backup, record the storage class name. And during restoring and creating PV/PVC, let the user choose which StorageClass name to use while selecting the recorded as default. That would be the best of both alternatives.

Longhorn uses the default Longhorn Static StorageClass Name longhorn-static when PV/PVC is created.

And yes, we could improve this one with solutions:

  • During backup, record the storage class name. And during restore, use the storage class name found in the backup metadata. It’s good for a bunch of backups to restore at one time.
  • During restoring and creating PV/PVC, let the user choose which StorageClass name to use. This provides more granularity to choose different storage class names per volume.

Hi @roger-ryao After discussed with @innobead , we think there is no need to add this validation. But we will add a note in the document Thanks

Here is the doc PR: https://github.com/longhorn/website/pull/729

@ChanYiLin PRs needs to update to fix the conflicts.

Hi @smallteeths This issue needs UI support and the milestone is 1.5.0, thanks!

So we are adding 1 new field when creating PV/PVC in volume page

  • storageClassName: string; value: user define, but can be empty

May need a ? for explaining the field

If the value is not given, 
Longhorn first use the storageClassName stored in backup volume 
if the volume is restored from a backup. 
Otherwise, Longhorn use the storageClassName in default setting

Thanks!