Bug #49582
openceph-volume lvm batch does not support valid LV passed by 'dm' names
0%
Description
When passing block like /dev/dm-0 (a valid LV):
# pvs PV VG Fmt Attr PSize PFree /dev/vdb leseb lvm2 a-- <30.00g 0 # vgs VG #PV #LV #SN Attr VSize VFree leseb 1 1 0 wz--n- <30.00g 0 # lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert ceph leseb -wi-a----- <30.00g # dmsetup ls leseb-ceph (252:0)
c-v somehow fails to configure it:
2021-03-03 14:32:11.330974 D | exec: Running command: stdbuf -oL ceph-volume --log-path /tmp/ceph-log lvm batch --prepare --bluestore --yes --osds-per-device 1 /dev/dm-0 --report
2021-03-03 14:32:11.661471 D | exec: --> DEPRECATION NOTICE
2021-03-03 14:32:11.661524 D | exec: --> You are using the legacy automatic disk sorting behavior
2021-03-03 14:32:11.661531 D | exec: --> The Pacific release will change the default to --no-auto
2021-03-03 14:32:11.661535 D | exec: --> passed data devices: 0 physical, 1 LVM
2021-03-03 14:32:11.661539 D | exec: --> relative data size: 1.0
2021-03-03 14:32:11.663210 D | exec: Traceback (most recent call last):
2021-03-03 14:32:11.663237 D | exec: File "/usr/sbin/ceph-volume", line 11, in <module>
2021-03-03 14:32:11.663242 D | exec: load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')()
2021-03-03 14:32:11.663247 D | exec: File "/usr/lib/python3.6/site-packages/ceph_volume/main.py", line 40, in __init__
2021-03-03 14:32:11.663251 D | exec: self.main(self.argv)
2021-03-03 14:32:11.663254 D | exec: File "/usr/lib/python3.6/site-packages/ceph_volume/decorators.py", line 59, in newfunc
2021-03-03 14:32:11.663258 D | exec: return f(*a, **kw)
2021-03-03 14:32:11.663266 D | exec: File "/usr/lib/python3.6/site-packages/ceph_volume/main.py", line 152, in main
2021-03-03 14:32:11.663270 D | exec: terminal.dispatch(self.mapper, subcommand_args)
2021-03-03 14:32:11.663274 D | exec: File "/usr/lib/python3.6/site-packages/ceph_volume/terminal.py", line 194, in dispatch
2021-03-03 14:32:11.663277 D | exec: instance.main()
2021-03-03 14:32:11.663281 D | exec: File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/main.py", line 42, in main
2021-03-03 14:32:11.663285 D | exec: terminal.dispatch(self.mapper, self.argv)
2021-03-03 14:32:11.663289 D | exec: File "/usr/lib/python3.6/site-packages/ceph_volume/terminal.py", line 194, in dispatch
2021-03-03 14:32:11.663292 D | exec: instance.main()
2021-03-03 14:32:11.663297 D | exec: File "/usr/lib/python3.6/site-packages/ceph_volume/decorators.py", line 16, in is_root
2021-03-03 14:32:11.663300 D | exec: return func(*a, **kw)
2021-03-03 14:32:11.663332 D | exec: File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/batch.py", line 402, in main
2021-03-03 14:32:11.663344 D | exec: plan = self.get_plan(self.args)
2021-03-03 14:32:11.663348 D | exec: File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/batch.py", line 440, in get_plan
2021-03-03 14:32:11.663352 D | exec: args.wal_devices)
2021-03-03 14:32:11.663355 D | exec: File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/batch.py", line 460, in get_deployment_layout
2021-03-03 14:32:11.663359 D | exec: plan.extend(get_lvm_osds(lvm_devs, args))
2021-03-03 14:32:11.663363 D | exec: File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/batch.py", line 93, in get_lvm_osds
2021-03-03 14:32:11.663366 D | exec: disk.Size(b=int(lv.lvs[0].lv_size)),
2021-03-03 14:32:11.663370 D | exec: IndexError: list index out of range
failed to configure devices: failed to initialize devices: failed ceph-volume report: exit status 1
Updated by David Mandelberg about 3 years ago
I looked into this a bit and I think I see how to fix it. I'll try to send a PR at some point.
It looks like https://github.com/ceph/ceph/blob/8f57c508df7255bcecbb072d622570c337fdedaa/src/ceph-volume/ceph_volume/util/device.py#L159-L164 is where ceph-volume converts a string argument to info about an LV. When passed /dev/dm-0, that's calling a command like lvs -a -S lv_path=/dev/dm-0 (with some extra args that aren't really relevant here I think). The lv_path attribute isn't expecting a dm-X device path though, so it doesn't match the device. I ran lvs -S help to see the available attributes, and I don't see anything that looks like it could match /dev/dm-0, but this works on the command line:
root@k8sc0n0:~# stat /dev/dm-0 File: /dev/dm-0 Size: 0 Blocks: 0 IO Block: 4096 block special file Device: 5h/5d Inode: 247 Links: 1 Device type: fe,0 Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) Access: 2021-04-13 13:58:41.754711923 -0400 Modify: 2021-04-13 13:58:41.754711923 -0400 Change: 2021-04-13 13:58:41.754711923 -0400 Birth: - root@k8sc0n0:~# lvs -a -S lv_kernel_major=254,lv_kernel_minor=0 LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert root k8sc0n0-vg -wi-ao---- 64.00g
That combination of stat and filtering on the kernel device numbers should be easy enough to do in Python too, I think.
Updated by David Mandelberg about 3 years ago
It looks like I was wrong about which flags are relevant. With the --readonly flag, lv_kernel_major and lv_kernel_minor are both -1, so I don't think we can use them here. I'll work on a PR that searches for symlinks and matches on one of the path fields.
Updated by Ruben Garcia over 2 years ago
I have a similar issue with
ceph version 14.2.21 (5ef401921d7a88aea18ec7558f7f9374ebd8f5a6) nautilus (stable)
This is the latest debian stable currently.
Running
ceph-volume --cluster ceph lvm batch --bluestore DEVICES --report --format=json
for multipath devices, gives
--> IndexError: list index out of range
Here are some further details:
https://github.com/ceph/ceph-ansible/issues/6890
In fact, testing multiple devices (disk, lvm, multipath), I was not able to get ceph-volume to accept any of them.
Updated by Ruben Garcia over 2 years ago
Ruben Garcia wrote:
I have a similar issue with
ceph version 14.2.21 (5ef401921d7a88aea18ec7558f7f9374ebd8f5a6) nautilus (stable)
This is the latest debian stable currently.
Running
ceph-volume --cluster ceph lvm batch --bluestore DEVICES --report --format=json
for multipath devices, gives
--> IndexError: list index out of rangeHere are some further details:
https://github.com/ceph/ceph-ansible/issues/6890In fact, testing multiple devices (disk, lvm, dm, multipath), I was not able to get ceph-volume to accept any of them.