ceph-csi: Missing cephfs volume metrics

Describe the bug

I am not able to see usage stats for cephfs volumes, only those for rbd - kubelet_volume_stats_used_bytes.

Environment details

  • Image/version of Ceph CSI driver : 3.0
  • Helm chart version : not installed via helm
  • Kernel version : 4.15.0-112-generic
  • Mounter used for mounting PVC (for cephfs its fuse or kernel. for rbd its krbd or rbd-nbd) : fuse
  • Kubernetes cluster version : 1.18.5
  • Ceph cluster version : 15.2.4

Actual results

Metrics only available for rbd volumes.

kubelet_volume_stats_used_bytes{endpoint="https-metrics",instance="X.X.X.131:10250",job="kubelet",namespace="my_ns",node="node-01",persistentvolumeclaim="pvc0",service="prometheus-operator-kubelet"} | 46190592
kubelet_volume_stats_used_bytes{endpoint="https-metrics",instance="X.X.X.73:10250",job="kubelet",namespace="my_ns",node="node-06",persistentvolumeclaim="pvc1",service="prometheus-operator-kubelet"} | 46182400
kubelet_volume_stats_used_bytes{endpoint="https-metrics",instance="X.X.X.216:10250",job="kubelet",namespace="my_ns",node="node-03",persistentvolumeclaim="pvc2",service="prometheus-operator-kubelet"} | 46534656
kubelet_volume_stats_used_bytes{endpoint="https-metrics",instance="X.X.X.201:10250",job="kubelet",namespace="my_ns",node="node-2",persistentvolumeclaim="pvc3",service="prometheus-operator-kubelet"} | 7071977472
kubelet_volume_stats_used_bytes{endpoint="https-metrics",instance="X.X.X.201:10250",job="kubelet",namespace="my_ns",node="node-2",persistentvolumeclaim="pvc4",service="prometheus-operator-kubelet"} | 2629632

Expected behavior

Metrics available for ceph volumes as well.

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 1
  • Comments: 25 (2 by maintainers)

Most upvoted comments

@eachirei metrics like kubelet_volume_stats_* are only available if the volume is in use by a Pod. Kubelet gathers these metrics, and can only access the Volume when it is in use/mounted.

Could you let us know if this additional detail helps?