Bug #59640: can't create new OSD with `lvm batch` when using pre-created VGs/LVs - ceph-volume - Ceph

Actions

Copy link

Bug #59640

closed

can't create new OSD with `lvm batch` when using pre-created VGs/LVs

Added by Guillaume Abrioux 12 months ago. Updated 9 months ago.

Status:

Resolved

Priority:

Normal

Assignee:

Guillaume Abrioux

Target version:

% Done:

Source:

Tags:

backport_processed

Backport:

reef,quincy,pacific

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

51343

Crash signature (v1):

Crash signature (v2):

Description

initially 8 OSD were deployed with the following command:


# ceph-volume lvm batch --prepare --bluestore --no-systemd /dev/data-vg-vdb/data-lv-vdb /dev/data-vg-vdc/data-lv-vdc /dev/data-vg-vdd/data-lv-vdd /dev/data-vg-vde/data-lv-vde /dev/data-vg-vdf/data-lv-vdf /dev/data-vg-vdg/data-lv-vdg /dev/data-vg-vdh/data-lv-vdh /dev/data-vg-vdi/data-lv-vdi --db-devices /dev/db-vg1/db-lv-vdb /dev/db-vg1/db-lv-vdc /dev/db-vg1/db-lv-vdd /dev/db-vg1/db-lv-vde /dev/db-vg1/db-lv-vdf /dev/db-vg1/db-lv-vdg /dev/db-vg1/db-lv-vdh /dev/db-vg1/db-lv-vdi --wal-devices /dev/wal-vg1/wal-lv-vdb /dev/wal-vg1/wal-lv-vdc /dev/wal-vg1/wal-lv-vdd /dev/wal-vg1/wal-lv-vde /dev/wal-vg1/wal-lv-vdf /dev/wal-vg1/wal-lv-vdg /dev/wal-vg1/wal-lv-vdh /dev/wal-vg1/wal-lv-vdi

then, it fails when trying to add a new OSD from `/dev/data-vg-vdk/data-lv-vdk` as block device, `/dev/db-vg1/db-lv-vdk` as db device and `/dev/wal-vg1/wal-lv-vdk` as wal device with the following command:


# ceph-volume lvm batch --prepare --bluestore --no-systemd /dev/data-vg-vdb/data-lv-vdb /dev/data-vg-vdc/data-lv-vdc /dev/data-vg-vdd/data-lv-vdd /dev/data-vg-vde/data-lv-vde /dev/data-vg-vdf/data-lv-vdf /dev/data-vg-vdg/data-lv-vdg /dev/data-vg-vdh/data-lv-vdh /dev/data-vg-vdi/data-lv-vdi /dev/data-vg-vdk/data-lv-vdk --db-devices /dev/db-vg1/db-lv-vdb /dev/db-vg1/db-lv-vdc /dev/db-vg1/db-lv-vdd /dev/db-vg1/db-lv-vde /dev/db-vg1/db-lv-vdf /dev/db-vg1/db-lv-vdg /dev/db-vg1/db-lv-vdh /dev/db-vg1/db-lv-vdi /dev/db-vg1/db-lv-vdk --wal-devices /dev/wal-vg1/wal-lv-vdb /dev/wal-vg1/wal-lv-vdc /dev/wal-vg1/wal-lv-vdd /dev/wal-vg1/wal-lv-vde /dev/wal-vg1/wal-lv-vdf /dev/wal-vg1/wal-lv-vdg /dev/wal-vg1/wal-lv-vdh /dev/wal-vg1/wal-lv-vdi /dev/wal-vg1/wal-lv-vdk --report
--> passed data devices: 0 physical, 9 LVM
--> relative data size: 1.0
--> passed block_db devices: 0 physical, 9 LVM
--> 9 fast allocations != 1 num_osds

Related issues 3 (0 open — 3 closed)

Actions

Copy link

Updated by Guillaume Abrioux 12 months ago

`get_lvm_fast_allocs()` in `devices/lvm/batch.py` calls the property `Device.used_by_ceph` in order to filter out devices that are already used by ceph.
The issue is that `Device.used_by_ceph()` itself filters out journal devices (db/wal) given that a db/wal device can be shared between multiple OSDs.
The consequence is that `Device.used_by_ceph()` always returns False for a db/wal device (even if it is actually already used by ceph) so `get_lvm_fast_allocs()` always returns the full list of the passed db/wal devices on the `lvm batch` CLI command.
Finally, the logic in `devices.lvm.batch.get_deployment_layout()` checks whether the length of the list returned by `get_lvm_fast_allocs()` is equal to `num_osds` (the number of OSD being created), if not it fails.

Actions

Copy link