Bug #62593
openceph-bluestore-tool bluefs-bdev-new-db creates block.db symlink to non-persistent /dev/dm-## instead of given and stable /dev/VG_NMAE/LV_NAME path
0%
Description
I added a dedicated DB device to an existing OSD and then found the auto-created symlink for block.db to not point to the device path as I gave it.
The DB device in my case is actually a LV on a VG "ceph-journal" with a flash device as its PV. This is to allow for multiple OSD to use a single SSD, quite a common approach I believe.
1) So I created a LV via:
lvcreate -L 200G --name block.db-osd1 ceph-journal
2) I then shutdown the OSD and migrated the DB to the new device / LV:
systemctl stop ceph-osd@1.service ceph-bluestore-tool bluefs-bdev-new-db --path /var/lib/ceph/osd/ceph-1 --dev-target /dev/ceph-journal/block.db-osd1
When looking at /var/lib/ceph/osd/ceph-1 I saw that the new block.db symlink pointed to /dev/dm-53.
This of course is not stable across reboots.
Updated by Igor Fedotov 8 months ago
This looks like an expected behavior to me. One has to use ceph-volume wrapper on top ceph-bluestore-tool to properly update lv tags and enable proper symlink recreation on reboot this way.
Please see relevant tickets for more information:
https://tracker.ceph.com/issues/42928 - similar issue
https://tracker.ceph.com/issues/49554 - ceph-volume getting the feature
Updated by Igor Fedotov 8 months ago
- Is duplicate of Bug #42928: ceph-bluestore-tool bluefs-bdev-new-db does not update lv tags added
Updated by Christian Rohmann 8 months ago
Igor thanks for looking into this!
I don't quite see this as real duplicate / invalid:
In my case the OSD was not created by ceph-volume, that's why I used ceph-bluestore-tool directly. And isn't it "ceph-bluestore-tool" that creates the block.db symlink for the OSD in the end? Even if wrapped in / called by ceph-volume it should point to the given the target.
Or how am I holding this wrong?