Bug #59640
closedcan't create new OSD with `lvm batch` when using pre-created VGs/LVs
0%
Description
initially 8 OSD were deployed with the following command:
# ceph-volume lvm batch --prepare --bluestore --no-systemd /dev/data-vg-vdb/data-lv-vdb /dev/data-vg-vdc/data-lv-vdc /dev/data-vg-vdd/data-lv-vdd /dev/data-vg-vde/data-lv-vde /dev/data-vg-vdf/data-lv-vdf /dev/data-vg-vdg/data-lv-vdg /dev/data-vg-vdh/data-lv-vdh /dev/data-vg-vdi/data-lv-vdi --db-devices /dev/db-vg1/db-lv-vdb /dev/db-vg1/db-lv-vdc /dev/db-vg1/db-lv-vdd /dev/db-vg1/db-lv-vde /dev/db-vg1/db-lv-vdf /dev/db-vg1/db-lv-vdg /dev/db-vg1/db-lv-vdh /dev/db-vg1/db-lv-vdi --wal-devices /dev/wal-vg1/wal-lv-vdb /dev/wal-vg1/wal-lv-vdc /dev/wal-vg1/wal-lv-vdd /dev/wal-vg1/wal-lv-vde /dev/wal-vg1/wal-lv-vdf /dev/wal-vg1/wal-lv-vdg /dev/wal-vg1/wal-lv-vdh /dev/wal-vg1/wal-lv-vdi
then, it fails when trying to add a new OSD from `/dev/data-vg-vdk/data-lv-vdk` as block device, `/dev/db-vg1/db-lv-vdk` as db device and `/dev/wal-vg1/wal-lv-vdk` as wal device with the following command:
# ceph-volume lvm batch --prepare --bluestore --no-systemd /dev/data-vg-vdb/data-lv-vdb /dev/data-vg-vdc/data-lv-vdc /dev/data-vg-vdd/data-lv-vdd /dev/data-vg-vde/data-lv-vde /dev/data-vg-vdf/data-lv-vdf /dev/data-vg-vdg/data-lv-vdg /dev/data-vg-vdh/data-lv-vdh /dev/data-vg-vdi/data-lv-vdi /dev/data-vg-vdk/data-lv-vdk --db-devices /dev/db-vg1/db-lv-vdb /dev/db-vg1/db-lv-vdc /dev/db-vg1/db-lv-vdd /dev/db-vg1/db-lv-vde /dev/db-vg1/db-lv-vdf /dev/db-vg1/db-lv-vdg /dev/db-vg1/db-lv-vdh /dev/db-vg1/db-lv-vdi /dev/db-vg1/db-lv-vdk --wal-devices /dev/wal-vg1/wal-lv-vdb /dev/wal-vg1/wal-lv-vdc /dev/wal-vg1/wal-lv-vdd /dev/wal-vg1/wal-lv-vde /dev/wal-vg1/wal-lv-vdf /dev/wal-vg1/wal-lv-vdg /dev/wal-vg1/wal-lv-vdh /dev/wal-vg1/wal-lv-vdi /dev/wal-vg1/wal-lv-vdk --report
--> passed data devices: 0 physical, 9 LVM
--> relative data size: 1.0
--> passed block_db devices: 0 physical, 9 LVM
--> 9 fast allocations != 1 num_osds
Updated by Guillaume Abrioux 12 months ago
`get_lvm_fast_allocs()` in `devices/lvm/batch.py` calls the property `Device.used_by_ceph` in order to filter out devices that are already used by ceph.
The issue is that `Device.used_by_ceph()` itself filters out journal devices (db/wal) given that a db/wal device can be shared between multiple OSDs.
The consequence is that `Device.used_by_ceph()` always returns False for a db/wal device (even if it is actually already used by ceph) so `get_lvm_fast_allocs()` always returns the full list of the passed db/wal devices on the `lvm batch` CLI command.
Finally, the logic in `devices.lvm.batch.get_deployment_layout()` checks whether the length of the list returned by `get_lvm_fast_allocs()` is equal to `num_osds` (the number of OSD being created), if not it fails.
Updated by Guillaume Abrioux 12 months ago
- Status changed from In Progress to Pending Backport
Updated by Backport Bot 12 months ago
- Copied to Backport #61152: reef: can't create new OSD with `lvm batch` when using pre-created VGs/LVs added
Updated by Backport Bot 12 months ago
- Copied to Backport #61153: quincy: can't create new OSD with `lvm batch` when using pre-created VGs/LVs added
Updated by Backport Bot 12 months ago
- Copied to Backport #61154: pacific: can't create new OSD with `lvm batch` when using pre-created VGs/LVs added
Updated by Guillaume Abrioux 10 months ago
- Status changed from Pending Backport to Resolved