copied from downstream:
It turns out ceph-volume fails here (although this behavior hasn't been reported upstream) to create OSDs because during the creation workflow a call to `restorecon` fails [1].
As a consequence, ceph-volume leaves an OSD not fully prepared, the cluster won't even see it which means the osd ID that was going to be used will be picked by another OSD later.
That being said, when running from a container, ceph-volume shouldn't see SELinux is enabled [2], so it shouldn't make this call to `restorecon` [3].
The fix I would suggest here is to make cephadm pass en environment variable `CEPH_VOLUME_SKIP_RESTORECON` [4] to skip this operation since it doesn't make sense to run it in a containerized context, see PR [5]
[1] restorecon failure:
[2021-11-24 09:37:54,441][ceph_volume.process][INFO ] Running command: /usr/sbin/selinuxenabled
[2021-11-24 09:37:54,444][ceph_volume.process][INFO ] Running command: /usr/sbin/restorecon /var/lib/ceph/osd/ceph-1
[2021-11-24 09:37:54,462][ceph_volume.process][INFO ] stderr No such file or directory
[2021-11-24 09:37:54,462][ceph_volume.devices.lvm.prepare][ERROR ] lvm prepare was unable to complete
[2] expected behavior:
[2021-11-24 09:39:48,605][ceph_volume.process][INFO ] Running command: /usr/sbin/selinuxenabled
[2021-11-24 09:39:48,608][ceph_volume.util.system][INFO ] SELinux is not enabled, will not call restorecon
[2021-11-24 09:39:48,609][ceph_volume.process][INFO ] Running command: /usr/bin/chown -h ceph:ceph /dev/ceph-fsid/osd-block-osdid