Bug #22305
closedceph-volume should be able to handle multiple LVM (VG/LV) tags
0%
Description
ceph-volume lvm prepare --bluestore --data osd.9/osd.9 [2017-12-01 17:25:25,279][ceph_volume.process][INFO ] Running command: ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new 063c7de3-d4b2-463b-9f56-7a76b0b48197 [2017-12-01 17:25:25,940][ceph_volume.process][INFO ] stdout 25 [2017-12-01 17:25:25,940][ceph_volume.process][INFO ] Running command: sudo lvs --noheadings --separator=";" -o lv_tags,lv_path,lv_name,vg_name,lv_uuid [2017-12-01 17:25:25,977][ceph_volume.process][INFO ] stdout ";"/dev/LVM0/CEPH";"CEPH";"LVM0";"y4Al1c-SFHH-VARl-XQf3-Qsc8-H3MN-LLIIj4 [2017-12-01 17:25:25,978][ceph_volume.process][INFO ] stdout ";"/dev/LVM0/ROOT";"ROOT";"LVM0";"31V3cd-E2b1-LcDz-2loq-egvh-lz4e-3u20ZN [2017-12-01 17:25:25,978][ceph_volume.process][INFO ] stdout ";"/dev/LVM0/SWAP";"SWAP";"LVM0";"hI3cNL-sddl-yXFB-BOXT-5R6j-fDtZ-kNixYa [2017-12-01 17:25:25,978][ceph_volume.process][INFO ] stdout d77bfa9f-4d8d-40df-852a-692a94929ed2";"/dev/osd.9/osd.9";"osd.9";"osd.9";"3NAmK8-U3Fx-KUOm-f8x8-aEtO-MbYh-uPGHhR [2017-12-01 17:25:25,979][ceph_volume][ERROR ] exception caught by decorator Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/ceph_volume/decorators.py", line 59, in newfunc return f(*a, **kw) File "/usr/lib/python2.7/dist-packages/ceph_volume/main.py", line 144, in main terminal.dispatch(self.mapper, subcommand_args) File "/usr/lib/python2.7/dist-packages/ceph_volume/terminal.py", line 131, in dispatch instance.main() File "/usr/lib/python2.7/dist-packages/ceph_volume/devices/lvm/main.py", line 38, in main terminal.dispatch(self.mapper, self.argv) File "/usr/lib/python2.7/dist-packages/ceph_volume/terminal.py", line 131, in dispatch instance.main() File "/usr/lib/python2.7/dist-packages/ceph_volume/devices/lvm/prepare.py", line 293, in main self.prepare(args) File "/usr/lib/python2.7/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python2.7/dist-packages/ceph_volume/devices/lvm/prepare.py", line 206, in prepare block_lv = self.get_lv(args.data) File "/usr/lib/python2.7/dist-packages/ceph_volume/devices/lvm/prepare.py", line 102, in get_lv return api.get_lv(lv_name=lv_name, vg_name=vg_name) File "/usr/lib/python2.7/dist-packages/ceph_volume/api/lvm.py", line 162, in get_lv lvs = Volumes() File "/usr/lib/python2.7/dist-packages/ceph_volume/api/lvm.py", line 411, in __init__ self._populate() File "/usr/lib/python2.7/dist-packages/ceph_volume/api/lvm.py", line 416, in _populate self.append(Volume(**lv_item)) File "/usr/lib/python2.7/dist-packages/ceph_volume/api/lvm.py", line 638, in __init__ self.tags = parse_tags(kw['lv_tags']) File "/usr/lib/python2.7/dist-packages/ceph_volume/api/lvm.py", line 66, in parse_tags key, value = tag_assignment.split('=', 1) ValueError: need more than 1 value to unpack
Additional info:
...
--- Volume group ---
VG Name osd.9
...
--- Logical volume ---
LV Path /dev/osd.9/osd.9
LV Name osd.9
VG Name osd.9
...
lvs -o vg_tags /dev/osd.9/osd.9
VG Tags
d77bfa9f-4d8d-40df-852a-692a94929ed2
...
lvs -o lv_tags /dev/osd.9/osd.9
LV Tags
d77bfa9f-4d8d-40df-852a-692a94929ed2
Use case:
Just like ceph-volume we use LVM tags to identify a know UUID (which corresponds to a disk in our setup).
Alfredo Deza commented on this mailinglist thread:
"Looks like there is a tag in there that broke it. Lets follow up on a
tracker issue so that we don't hijack this thread?"
Ideally ceph-volume is able to handle multiple (already existing) tags. LVM VG / LV supports multiple tags.
Updated by Stefan Kooman over 6 years ago
Removing VG / LV tags:
# vgchange osd.9 --deltag d77bfa9f-4d8d-40df-852a-692a94929ed2
Volume group "osd.9" successfully changed
# lvchange osd.9 --deltag d77bfa9f-4d8d-40df-852a-692a94929ed2
Logical volume "osd.9" changed.
Now ceph-volume succesfully finishes:
~# ceph-volume lvm prepare --bluestore --data osd.9/osd.9
Running command: sudo mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-26
Running command: chown -R ceph:ceph /dev/dm-1
Running command: sudo ln -s /dev/osd.9/osd.9 /var/lib/ceph/osd/ceph-26/block
Running command: sudo ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-26/activate.monmap
stderr: got monmap epoch 19
Running command: ceph-authtool /var/lib/ceph/osd/ceph-26/keyring --create-keyring --name osd.26 --add-key AQBC5yNamF7rBBAA7OlQFL+FsnMEpnNGq5fhrg==
stdout: creating /var/lib/ceph/osd/ceph-26/keyring
added entity osd.26 auth auth(auid = 18446744073709551615 key=AQBC5yNamF7rBBAA7OlQFL+FsnMEpnNGq5fhrg== with 0 caps)
Running command: chown -R ceph:ceph /var/lib/ceph/osd/ceph-26/keyring
Running command: chown -R ceph:ceph /var/lib/ceph/osd/ceph-26/
Running command: sudo ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 26 --monmap /var/lib/ceph/osd/ceph-26/activate.monmap --key ******************************** --osd-data /var/lib/ceph/osd/ceph-26/ --osd-uuid 69ec1761-d235-469e-abe2-e15744b21330 --setuser ceph --setgroup ceph
stderr: 2017-12-03 13:00:04.944344 7fa2211d4e00 -1 bluestore(/var/lib/ceph/osd/ceph-26//block) _read_bdev_label unable to decode label at offset 102: buffer::malformed_input: void bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) decode past end of struct encoding
stderr: 2017-12-03 13:00:04.944699 7fa2211d4e00 -1 bluestore(/var/lib/ceph/osd/ceph-26//block) _read_bdev_label unable to decode label at offset 102: buffer::malformed_input: void bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) decode past end of struct encoding
2017-12-03 13:00:04.944959 7fa2211d4e00 -1 bluestore(/var/lib/ceph/osd/ceph-26//block) _read_bdev_label unable to decode label at offset 102: buffer::malformed_input: void bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) decode past end of struct encoding
stderr: 2017-12-03 13:00:04.945947 7fa2211d4e00 -1 bluestore(/var/lib/ceph/osd/ceph-26/) _read_fsid unparsable uuid
stderr: 2017-12-03 13:00:06.954608 7fa2211d4e00 -1 key AQBC5yNamF7rBBAA7OlQFL+FsnMEpnNGq5fhrg==
stderr: 2017-12-03 13:00:07.503944 7fa2211d4e00 -1 created object store /var/lib/ceph/osd/ceph-26/ for osd.26 fsid 918084f6-66ff-4f47-8feb-e86b7b393200
root@ceph03:~# lvs -o vg_tags /dev/osd.9/osd.9
VG Tags
root@ceph03:~# lvs -o lv_tags /dev/osd.9/osd.9
LV Tags
ceph.block_device=/dev/osd.9/osd.9,ceph.block_uuid=3NAmK8-U3Fx-KUOm-f8x8-aEtO-MbYh-uPGHhR,ceph.cluster_fsid=918084f6-66ff-4f47-8feb-e86b7b393200,ceph.cluster_name=ceph,ceph.osd_fsid=69ec1761-d235-469e-abe2-e15744b21330,ceph.osd_id=26,ceph.type=block
Apperently ceph-volume only uses lv tags and no vg tags.
Updated by Alfredo Deza over 6 years ago
- Status changed from New to 12
- Priority changed from Normal to High
We already have a fix for this, should have a pull request out soon. Thank you for reporting it!
Updated by Alfredo Deza over 6 years ago
- Status changed from 12 to Fix Under Review
Updated by Alfredo Deza over 6 years ago
- Status changed from Fix Under Review to Resolved
merged commit f956c6f into master
Updated by Alfredo Deza over 6 years ago
Luminous backport https://github.com/ceph/ceph/pull/19528