rook: c-v raw fails to list existing OSD (blocks CI)
It seems that ceph-volume fails to list existing raw osd, logs failure example:
2020-08-05 18:29:44.560689 D | exec: Running command: stdbuf -oL ceph-volume --log-path /tmp/ceph-log raw list /mnt/set1-data-0-b2jvl --format json
2020-08-05 18:29:45.144142 E | cephosd: Traceback (most recent call last):
File "/usr/sbin/ceph-volume", line 9, in <module>
load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')()
File "/usr/lib/python2.7/site-packages/ceph_volume/main.py", line 39, in __init__
self.main(self.argv)
File "/usr/lib/python2.7/site-packages/ceph_volume/decorators.py", line 59, in newfunc
return f(*a, **kw)
File "/usr/lib/python2.7/site-packages/ceph_volume/main.py", line 150, in main
terminal.dispatch(self.mapper, subcommand_args)
File "/usr/lib/python2.7/site-packages/ceph_volume/terminal.py", line 194, in dispatch
instance.main()
File "/usr/lib/python2.7/site-packages/ceph_volume/devices/raw/main.py", line 32, in main
terminal.dispatch(self.mapper, self.argv)
File "/usr/lib/python2.7/site-packages/ceph_volume/terminal.py", line 194, in dispatch
instance.main()
File "/usr/lib/python2.7/site-packages/ceph_volume/devices/raw/list.py", line 110, in main
self.list(args)
File "/usr/lib/python2.7/site-packages/ceph_volume/decorators.py", line 16, in is_root
return func(*a, **kw)
File "/usr/lib/python2.7/site-packages/ceph_volume/devices/raw/list.py", line 66, in list
report = self.generate(args.device)
File "/usr/lib/python2.7/site-packages/ceph_volume/devices/raw/list.py", line 54, in generate
whoami = oj[dev]['whoami']
KeyError: 'whoami'
Somewhat the output of ceph-bluestore-tool show-label --dev is incomplete.
Every CI runs are affected by this bug.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 17 (15 by maintainers)
Commits related to this issue
- ceph: only use realPath on encryption The encryption scenario needs to use the /dev path instead of /mnt so that the udev call from c-v can succeed. Closes: https://github.com/rook/rook/issues/5996 ... — committed to leseb/rook by leseb 4 years ago
- ceph: only use realPath on encryption The encryption scenario needs to use the /dev path instead of /mnt so that the udev call from c-v can succeed. Closes: https://github.com/rook/rook/issues/5996 ... — committed to leseb/rook by leseb 4 years ago
- ceph: do not use realPath for osd on pvc It is fine to use /mnt/set1-data-0-k47mc, it looks like I thought there was an issue with udev but there is none... Closes: https://github.com/rook/rook/issu... — committed to leseb/rook by leseb 4 years ago
- ceph: do not use realPath for osd on pvc It is fine to use /mnt/set1-data-0-k47mc, it looks like I thought there was an issue with udev but there is none... Closes: https://github.com/rook/rook/issu... — committed to rook/rook by leseb 4 years ago
FYI I managed to catch it on my system:
systemd-udevd 1527 FLOCK WRITE 0 0 0 /dev...@satoru-takeuchi thanks for looking into this, yes I’m debugging here: https://github.com/rook/rook/pull/5977