ceph-csi: Failed to Mount PVC with RBD Map Failed Input/Output Error

Describe the bug

Hello everyone:

I met a problem when following the block storage example to create a MySQL Deployment.

Environment details

Image/version of Ceph CSI driver: quay.io/cephcsi/cephcsi:v3.3.1
Helm chart version: Not using Helm
Kernel version: kernel 4.1.0-25.el7.ucloud.x86_64
Mounter used for mounting PVC (for cephfs its fuse or kernel. for rbd its krbd or rbd-nbd): Sorry, I am not sure. I was following this example. Please tell me how to figure it out.
Kubernetes cluster version: 1.16.3
Ceph cluster version: quay.io/ceph/ceph:v16.2.5

os-release of my nodes:

NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

Steps to reproduce

Steps to reproduce the behavior:

Following this tutorial to install by rook.
Then following this one to create StorageClasses and a MySQL Deployment.

Actual results

I used the mysql.yaml to create a Pod, but failed with events:

Events:
  Type     Reason                  Age                   From                     Message
  ----     ------                  ----                  ----                     -------
  Warning  FailedScheduling        <unknown>             default-scheduler        pod has unbound immediate PersistentVolumeClaims (repeated 3 times)
  Warning  FailedScheduling        <unknown>             default-scheduler        pod has unbound immediate PersistentVolumeClaims (repeated 3 times)
  Normal   Scheduled               <unknown>             default-scheduler        Successfully assigned default/wordpress-mysql-b9ddd6d4c-7tzv2 to 10-0-0-8
  Normal   SuccessfulAttachVolume  38m                   attachdetach-controller  AttachVolume.Attach succeeded for volume "pvc-dfc6cb07-5334-4b7d-b733-9acea4e539f4"
  Warning  FailedMount             15m (x3 over 24m)     kubelet, 10-0-0-8        Unable to attach or mount volumes: unmounted volumes=[mysql-persistent-storage], unattached volumes=[default-token-zzhzg mysql-persistent-storage]: timed out waiting for the condition
  Warning  FailedMount             5m15s (x16 over 36m)  kubelet, 10-0-0-8        MountVolume.MountDevice failed for volume "pvc-dfc6cb07-5334-4b7d-b733-9acea4e539f4" : rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
  Warning  FailedMount  4m14s (x12 over 35m)  kubelet, 10-0-0-8  Unable to attach or mount volumes: unmounted volumes=[mysql-persistent-storage], unattached volumes=[mysql-persistent-storage default-token-zzhzg]: timed out waiting for the condition

Expected behavior

I expected the PVC can be mounted.

Logs

If the issue is in PVC mounting please attach complete logs of below containers.

csi-rbdplugin/csi-cephfsplugin and driver-registrar container logs from plugin pod from the node where the mount is failing.

csi-rbdplugin

Click to see logs

E0811 02:43:14.111059  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 02:44:15.317828  111893 rbd_attach.go:270] ID: 53 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:44:15.318074  111893 utils.go:171] ID: 53 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:44:15.952313  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 02:45:16.404865  111893 rbd_attach.go:270] ID: 56 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:45:16.405081  111893 utils.go:171] ID: 56 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:45:17.431068  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 02:46:17.982872  111893 rbd_attach.go:270] ID: 59 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:46:17.983039  111893 utils.go:171] ID: 59 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:46:20.029190  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 02:47:20.454037  111893 rbd_attach.go:270] ID: 62 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:47:20.454393  111893 utils.go:171] ID: 62 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:47:24.515306  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 02:48:24.931708  111893 rbd_attach.go:270] ID: 65 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:48:24.931889  111893 utils.go:171] ID: 65 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:48:33.016960  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 02:49:33.426659  111893 rbd_attach.go:270] ID: 68 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:49:33.426910  111893 utils.go:171] ID: 68 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:49:49.527140  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 02:50:49.901900  111893 rbd_attach.go:270] ID: 71 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:50:49.902093  111893 utils.go:171] ID: 71 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:51:22.255218  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 02:52:22.663508  111893 rbd_attach.go:270] ID: 75 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:52:22.663685  111893 utils.go:171] ID: 75 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:53:26.720425  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 02:54:27.136389  111893 rbd_attach.go:270] ID: 79 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:54:27.136576  111893 utils.go:171] ID: 79 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:56:29.246482  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 02:57:29.684725  111893 rbd_attach.go:270] ID: 84 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:57:29.684906  111893 utils.go:171] ID: 84 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 02:59:31.757912  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:00:32.166168  111893 rbd_attach.go:270] ID: 89 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:00:32.166352  111893 utils.go:171] ID: 89 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:02:34.260328  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:03:34.673863  111893 rbd_attach.go:270] ID: 94 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:03:34.674039  111893 utils.go:171] ID: 94 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:05:36.737569  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:06:37.101199  111893 rbd_attach.go:270] ID: 99 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:06:37.101442  111893 utils.go:171] ID: 99 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:08:39.182558  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:09:39.567305  111893 rbd_attach.go:270] ID: 104 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:09:39.567528  111893 utils.go:171] ID: 104 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:11:41.600815  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:12:41.984018  111893 rbd_attach.go:270] ID: 109 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:12:41.984211  111893 utils.go:171] ID: 109 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:14:44.050601  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:15:44.485580  111893 rbd_attach.go:270] ID: 114 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:15:44.485899  111893 utils.go:171] ID: 114 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:17:46.578109  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:18:46.986007  111893 rbd_attach.go:270] ID: 119 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:18:46.986281  111893 utils.go:171] ID: 119 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:20:49.052318  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:21:49.430177  111893 rbd_attach.go:270] ID: 124 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:21:49.430421  111893 utils.go:171] ID: 124 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:23:51.444137  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:24:51.927277  111893 rbd_attach.go:270] ID: 129 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:24:51.927445  111893 utils.go:171] ID: 129 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:26:53.971961  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:27:54.406427  111893 rbd_attach.go:270] ID: 134 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:27:54.406603  111893 utils.go:171] ID: 134 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:29:56.506276  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:30:56.891771  111893 rbd_attach.go:270] ID: 139 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:30:56.891992  111893 utils.go:171] ID: 139 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:32:58.929775  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:33:59.564231  111893 rbd_attach.go:270] ID: 144 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:33:59.564487  111893 utils.go:171] ID: 144 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:36:01.680602  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:37:02.078081  111893 rbd_attach.go:270] ID: 149 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:37:02.078302  111893 utils.go:171] ID: 149 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:39:04.098299  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:40:04.476777  111893 rbd_attach.go:270] ID: 154 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:40:04.477162  111893 utils.go:171] ID: 154 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:42:06.523343  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:43:06.950032  111893 rbd_attach.go:270] ID: 159 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:43:06.950253  111893 utils.go:171] ID: 159 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:45:09.032846  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:46:09.437932  111893 rbd_attach.go:270] ID: 164 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:46:09.438163  111893 utils.go:171] ID: 164 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:48:11.450567  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:49:11.857831  111893 rbd_attach.go:270] ID: 170 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:49:11.858010  111893 utils.go:171] ID: 170 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:51:13.941978  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:52:14.808588  111893 rbd_attach.go:270] ID: 175 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:52:14.808759  111893 utils.go:171] ID: 175 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:54:16.852556  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:55:17.216492  111893 rbd_attach.go:270] ID: 180 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:55:17.216651  111893 utils.go:171] ID: 180 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:57:19.301325  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 03:58:20.093199  111893 rbd_attach.go:270] ID: 185 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 03:58:20.093491  111893 utils.go:171] ID: 185 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:00:22.167753  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:01:22.540717  111893 rbd_attach.go:270] ID: 190 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:01:22.540899  111893 utils.go:171] ID: 190 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:03:24.623325  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:04:24.993003  111893 rbd_attach.go:270] ID: 195 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:04:24.993189  111893 utils.go:171] ID: 195 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:06:27.018872  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:07:27.456501  111893 rbd_attach.go:270] ID: 200 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:07:27.456686  111893 utils.go:171] ID: 200 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:09:29.496498  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:10:29.914898  111893 rbd_attach.go:270] ID: 205 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:10:29.915098  111893 utils.go:171] ID: 205 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:12:32.028086  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:13:32.470586  111893 rbd_attach.go:270] ID: 210 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:13:32.470774  111893 utils.go:171] ID: 210 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:15:34.593538  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:16:34.979168  111893 rbd_attach.go:270] ID: 215 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:16:34.979457  111893 utils.go:171] ID: 215 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:18:37.075155  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:19:37.586112  111893 rbd_attach.go:270] ID: 220 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:19:37.586354  111893 utils.go:171] ID: 220 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:21:39.659542  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:22:40.000998  111893 rbd_attach.go:270] ID: 225 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:22:40.001256  111893 utils.go:171] ID: 225 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:24:42.079758  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:25:42.523783  111893 rbd_attach.go:270] ID: 230 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:25:42.523942  111893 utils.go:171] ID: 230 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:27:44.595640  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:28:44.964132  111893 rbd_attach.go:270] ID: 235 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:28:44.964378  111893 utils.go:171] ID: 235 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:30:47.046670  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:31:47.436613  111893 rbd_attach.go:270] ID: 240 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:31:47.436854  111893 utils.go:171] ID: 240 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:33:49.495814  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:34:49.924057  111893 rbd_attach.go:270] ID: 245 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:34:49.924352  111893 utils.go:171] ID: 245 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:36:51.968525  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:37:52.332834  111893 rbd_attach.go:270] ID: 250 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:37:52.333003  111893 utils.go:171] ID: 250 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:39:54.419355  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:40:54.820603  111893 rbd_attach.go:270] ID: 255 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:40:54.820772  111893 utils.go:171] ID: 255 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:42:56.887753  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:43:57.275305  111893 rbd_attach.go:270] ID: 260 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:43:57.275510  111893 utils.go:171] ID: 260 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:45:59.318096  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:46:59.747168  111893 rbd_attach.go:270] ID: 265 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:46:59.747348  111893 utils.go:171] ID: 265 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:49:01.848539  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:50:02.277085  111893 rbd_attach.go:270] ID: 270 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:50:02.277298  111893 utils.go:171] ID: 270 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:52:04.363355  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:53:04.777959  111893 rbd_attach.go:270] ID: 275 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:53:04.778194  111893 utils.go:171] ID: 275 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:55:06.892276  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:56:07.294631  111893 rbd_attach.go:270] ID: 280 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:56:07.295018  111893 utils.go:171] ID: 280 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:58:09.339561  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 04:59:09.713483  111893 rbd_attach.go:270] ID: 285 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 04:59:09.713734  111893 utils.go:171] ID: 285 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:01:11.736563  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:02:12.127718  111893 rbd_attach.go:270] ID: 291 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:02:12.127922  111893 utils.go:171] ID: 291 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:04:14.180592  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:05:14.613636  111893 rbd_attach.go:270] ID: 296 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:05:14.613982  111893 utils.go:171] ID: 296 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:07:16.693948  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:08:17.046642  111893 rbd_attach.go:270] ID: 301 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:08:17.046850  111893 utils.go:171] ID: 301 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:10:19.090864  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:11:19.502481  111893 rbd_attach.go:270] ID: 306 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:11:19.502694  111893 utils.go:171] ID: 306 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:13:21.542869  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:14:21.910169  111893 rbd_attach.go:270] ID: 311 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:14:21.910443  111893 utils.go:171] ID: 311 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:16:23.940527  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:17:24.299761  111893 rbd_attach.go:270] ID: 316 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:17:24.299921  111893 utils.go:171] ID: 316 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:19:26.393572  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:20:26.843530  111893 rbd_attach.go:270] ID: 321 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:20:26.843758  111893 utils.go:171] ID: 321 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:22:28.963629  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:23:29.298118  111893 rbd_attach.go:270] ID: 326 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:23:29.298470  111893 utils.go:171] ID: 326 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:25:31.340815  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:26:31.749360  111893 rbd_attach.go:270] ID: 331 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:26:31.749545  111893 utils.go:171] ID: 331 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:28:33.858767  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:29:34.217132  111893 rbd_attach.go:270] ID: 336 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:29:34.217361  111893 utils.go:171] ID: 336 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:31:36.291572  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:32:36.629769  111893 rbd_attach.go:270] ID: 341 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:32:36.630052  111893 utils.go:171] ID: 341 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:34:38.734684  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:35:39.081296  111893 rbd_attach.go:270] ID: 346 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:35:39.081478  111893 utils.go:171] ID: 346 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:37:41.191577  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:38:41.584060  111893 rbd_attach.go:270] ID: 351 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:38:41.584298  111893 utils.go:171] ID: 351 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:40:43.697211  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:41:44.119394  111893 rbd_attach.go:270] ID: 356 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:41:44.119850  111893 utils.go:171] ID: 356 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:43:46.206910  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:44:46.559864  111893 rbd_attach.go:270] ID: 361 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:44:46.560039  111893 utils.go:171] ID: 361 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:46:48.644696  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:47:49.659709  111893 rbd_attach.go:270] ID: 366 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:47:49.659980  111893 utils.go:171] ID: 366 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:49:51.812372  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:50:52.556999  111893 rbd_attach.go:270] ID: 371 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:50:52.557168  111893 utils.go:171] ID: 371 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:52:54.573979  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:53:54.905313  111893 rbd_attach.go:270] ID: 376 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:53:54.905472  111893 utils.go:171] ID: 376 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:55:56.988824  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:56:57.432505  111893 rbd_attach.go:270] ID: 381 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:56:57.432666  111893 utils.go:171] ID: 381 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:58:59.487960  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 05:59:59.862473  111893 rbd_attach.go:270] ID: 386 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 05:59:59.862648  111893 utils.go:171] ID: 386 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:02:01.893291  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 06:03:02.240458  111893 rbd_attach.go:270] ID: 391 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:03:02.240810  111893 utils.go:171] ID: 391 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:05:04.340634  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 06:06:04.765431  111893 rbd_attach.go:270] ID: 396 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:06:04.765675  111893 utils.go:171] ID: 396 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:08:06.805365  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 06:09:07.175629  111893 rbd_attach.go:270] ID: 401 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:09:07.175788  111893 utils.go:171] ID: 401 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:11:09.204994  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 06:12:09.572942  111893 rbd_attach.go:270] ID: 406 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:12:09.573169  111893 utils.go:171] ID: 406 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:14:11.602214  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 06:15:12.037794  111893 rbd_attach.go:270] ID: 412 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:15:12.037984  111893 utils.go:171] ID: 412 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:17:14.116505  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 06:18:14.495350  111893 rbd_attach.go:270] ID: 417 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:18:14.495598  111893 utils.go:171] ID: 417 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:20:16.524607  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 06:21:17.356981  111893 rbd_attach.go:270] ID: 422 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:21:17.357203  111893 utils.go:171] ID: 422 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:23:19.467836  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 06:24:19.857444  111893 rbd_attach.go:270] ID: 427 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:24:19.857620  111893 utils.go:171] ID: 427 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:26:21.874635  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 06:27:22.243692  111893 rbd_attach.go:270] ID: 432 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:27:22.243868  111893 utils.go:171] ID: 432 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:29:24.295670  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 06:30:24.642486  111893 rbd_attach.go:270] ID: 437 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:30:24.642842  111893 utils.go:171] ID: 437 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:32:26.768449  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 06:33:27.109332  111893 rbd_attach.go:270] ID: 442 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:33:27.109535  111893 utils.go:171] ID: 442 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:35:29.153456  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 06:36:29.596089  111893 rbd_attach.go:270] ID: 447 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:36:29.596323  111893 utils.go:171] ID: 447 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:38:31.620447  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 06:39:32.058930  111893 rbd_attach.go:270] ID: 452 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:39:32.059090  111893 utils.go:171] ID: 452 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:41:34.151220  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 06:42:34.583022  111893 rbd_attach.go:270] ID: 457 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:42:34.583385  111893 utils.go:171] ID: 457 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:44:36.694893  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 06:45:37.108662  111893 rbd_attach.go:270] ID: 462 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:45:37.108847  111893 utils.go:171] ID: 462 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:47:39.193299  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 06:48:39.558290  111893 rbd_attach.go:270] ID: 467 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:48:39.558582  111893 utils.go:171] ID: 467 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:50:41.594510  111893 util.go:233] kernel 4.1.0-25.el7.ucloud.x86_64 does not support required features
W0811 06:51:41.995424  111893 rbd_attach.go:270] ID: 472 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd rbd: map error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error
E0811 06:51:41.995590  111893 utils.go:171] ID: 472 Req-ID: 0001-0009-rook-ceph-0000000000000002-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd GRPC error: rpc error: code = Internal desc = rbd: map failed with error an error (exit status 5) occurred while running rbd args: [--id csi-rbd-node -m 172.20.58.60:6789,172.20.85.112:6789,172.20.249.153:6789 --keyfile=***stripped*** map replicapool/csi-vol-d391ab80-fa4d-11eb-b4f0-0ac36fad62fd --device-type krbd --options noudev], rbd error output: rbd: sysfs write failed
rbd: map failed: (5) Input/output error

driver-registrar

I0811 01:54:07.733190  111768 main.go:113] Version: v2.2.0
I0811 01:54:08.744530  111768 node_register.go:52] Starting Registration Server at: /registration/rook-ceph.rbd.csi.ceph.com-reg.sock
I0811 01:54:08.744641  111768 node_register.go:61] Registration Server started at: /registration/rook-ceph.rbd.csi.ceph.com-reg.sock
I0811 01:54:08.744762  111768 node_register.go:83] Skipping healthz server because HTTP endpoint is set to: ""
I0811 01:54:51.584891  111768 main.go:80] Received GetInfo call: &InfoRequest{}
I0811 01:54:52.584854  111768 main.go:80] Received GetInfo call: &InfoRequest{}
I0811 01:54:52.826943  111768 main.go:90] Received NotifyRegistrationStatus call: &RegistrationStatus{PluginRegistered:true,Error:,}

Additional context

References

I found this issue may be similar to mine, but I am not sure.

Other versions

driver-registrar: k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0
csi: quay.io/cephcsi/cephcsi:v3.3.1

I am not familiar with Ceph, any suggestions are welcome! Thanks a lot!

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 15 (6 by maintainers)

Most upvoted comments

Hey @jtr109 , Can you run ceph -s in toolbox to get ceph status, by applying https://github.com/rook/rook/blob/master/cluster/examples/kubernetes/ceph/toolbox.yaml.

Also please send the cephblock pool, storageclass & pvc yamls you are using.

Rakshith-R on Aug 11, 2021

Hello everyone:

I update OS into CentOS 8 with kernel 4.18.0 and Install with the Octopus version. Everything works well!

Thanks all!

Thanks for confirming 😃

Yuggupta27 on Aug 19, 2021

What OS and kernel version do you use?

Here is my minikube setup:

$ hostnamectl 
   Static hostname: minikube
         Icon name: computer-vm
           Chassis: vm
        Machine ID: a9a805d140984f8381bc16acd812d2a0
           Boot ID: 49186ac4051b4c32b70e2d7564b927d6
    Virtualization: kvm
  Operating System: Buildroot 2020.02.12
            Kernel: Linux 4.19.182
      Architecture: x86-64

Thanks again for your help! I will retry soon!

jtr109 on Aug 16, 2021

@jtr109 also please provide

rbd info of the image
dmsg logs from the node where the map is failing

Madhu-1 on Aug 11, 2021