ceph-ansible: `"use ceph-volume to create {{ osd_objectstore }} osds"` is very slow, because it calls `ceph volume lvm list` once per OSD on a host
Bug Report
What happened: Running ceph-ansible on a Luminous / ceph-volume lvm / bluestore cluster takes a very long time, most of which is in the "use ceph-volume to create {{ osd_objectstore }} osds"
task. You can see below that it takes at least 2 hours per host - and this is not creating any OSDs, just going through the already-deployed ones on each host.
ceph-osd : use ceph-volume to create bluestore osds ------------------ 11228.03s
ceph-osd : use ceph-volume to create bluestore osds ------------------ 8466.27s
ceph-osd : use ceph-volume to create bluestore osds ------------------ 8390.08s
Looking at it, this is because the ceph-volume
plugin runs ceph volume lvm list
individually for each OSD, and that takes a while (between about 15 and 120s). It should, instead, call ceph volume lvm list
once, and process the output itself.
Environment:
- OS (e.g. from /etc/os-release): Ubuntu 16.04 and 18.04 (happens on both)
- Docker version if applicable (e.g.
docker version
): n/a - Ansible version (e.g.
ansible-playbook --version
): 2.6 - ceph-ansible version (e.g.
git head or tag or stable branch
): 3.2.3 - Ceph version (e.g.
ceph -v
): Luminous
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 19 (16 by maintainers)
@mcv21 Would you be able to test the ceph-volume fix provided by [1] ? The associated patch for Luminous is [2]. This will be part of the next 12.2.13 release.
[1] https://tracker.ceph.com/issues/37490 [2] https://github.com/ceph/ceph/pull/30094