Ceph » ceph-volume

Can we do something along these lines in ceph_volume/systemd/main.py after existing while loop?

1. using extra_data in ceph_volume/systemd/main.py, get ceph.wal_device and ceph.db_device from lvs tag with matching ceph.osd_id, ceph.osd_fsid, and type=block
2. e.g. where extra_data=ceph.osd_id=0-e20dbce0-34f4-46b3-8efc-f41edbcae3d7
   sudo lvs -o lv_tags|grep type=block | grep ceph.osd_id=0 | grep ceph\.osd_fsid=e20dbce0-34f4-46b3-8efc-f41edbcae3d7 | grep ceph\.wal_device
   sudo lvs -o lv_tags|grep type=block | grep ceph.osd_id=0 | grep ceph\.osd_fsid=e20dbce0-34f4-46b3-8efc-f41edbcae3d7 | grep ceph\.db_device
3. loop until the following is found or CEPH_VOLUME_SYSTEMD_TRIES times
4. where ceph.wal_device=/dev/ceph-wal-8a073a5b-6e42-43bf-a99d-e30c649362ea/osd-wal-e20dbce0-34f4-46b3-8efc-f41edbcae3d7
   sudo lvs -o lv_tags|grep type=wal | grep ceph.wal_device=/dev/ceph-wal-8a073a5b-6e42-43bf-a99d-e30c649362ea/osd-wal-e20dbce0-34f4-46b3-8efc-f41edbcae3d7
5. loop until the following is found or CEPH_VOLUME_SYSTEMD_TRIES times
6. where ceph.db_device=/dev/ceph-db-c37da146-b9a3-4339-bb2f-819f223982d3/osd-db-e20dbce0-34f4-46b3-8efc-f41edbcae3d7
   sudo lvs -o lv_tags|grep type=db | grep ceph.db_device=/dev/ceph-db-c37da146-b9a3-4339-bb2f-819f223982d3/osd-db-e20dbce0-34f4-46b3-8efc-f41edbcae3d7

Trying that again, the formatting in the last comment was unintended:

# using extra_data in ceph_volume/systemd/main.py, get ceph.wal_device and ceph.db_device from lvs tag with matching ceph.osd_id, ceph.osd_fsid, and type=block
# e.g. where extra_data=ceph.osd_id=0-e20dbce0-34f4-46b3-8efc-f41edbcae3d7
sudo lvs -o lv_tags|grep type=block | grep ceph.osd_id=0 | grep ceph\.osd_fsid=e20dbce0-34f4-46b3-8efc-f41edbcae3d7 | grep ceph\.wal_device
sudo lvs -o lv_tags|grep type=block | grep ceph.osd_id=0 | grep ceph\.osd_fsid=e20dbce0-34f4-46b3-8efc-f41edbcae3d7 | grep ceph\.db_device
# loop until the following is found or CEPH_VOLUME_SYSTEMD_TRIES times
# where ceph.wal_device=/dev/ceph-wal-8a073a5b-6e42-43bf-a99d-e30c649362ea/osd-wal-e20dbce0-34f4-46b3-8efc-f41edbcae3d7
sudo lvs -o lv_tags|grep type=wal | grep ceph.wal_device=/dev/ceph-wal-8a073a5b-6e42-43bf-a99d-e30c649362ea/osd-wal-e20dbce0-34f4-46b3-8efc-f41edbcae3d7
# loop until the following is found or CEPH_VOLUME_SYSTEMD_TRIES times
# where ceph.db_device=/dev/ceph-db-c37da146-b9a3-4339-bb2f-819f223982d3/osd-db-e20dbce0-34f4-46b3-8efc-f41edbcae3d7
sudo lvs -o lv_tags|grep type=db | grep ceph.db_device=/dev/ceph-db-c37da146-b9a3-4339-bb2f-819f223982d3/osd-db-e20dbce0-34f4-46b3-8efc-f41edbcae3d7

Pull request: https://github.com/ceph/ceph/pull/28357

Proposal pull request (from the work of coreycb) : https://github.com/ceph/ceph/pull/28520

In the coreycb proposal, the command was evaluated before waiting for WAL / DB devices to arrive.

In order to keep an equivalent timeout, I propose to put in the same loop

I do not know the consequences in case "ceph-volume simple" is used. I added a control "if sub_command == 'lvm':"

I commented in the PR, but want to reiterate here: we knew that there was a chance that in certain systems, the 30 tries at 5 second interval wouldn't be enough which is why we made it configurable and not hard coded.

In this case, this can benefit from changing the environment variables (as opposed to add extra intervals or tries).

The environment variables are:

CEPH_VOLUME_SYSTEMD_TRIES
CEPH_VOLUME_SYSTEMD_INTERVAL