Ceph » ceph-volume

If rbd devices are mapped on ceph cluster nodes (as they may be if you're running an iSCSI gateway for example), then ceph-volume inventory will list those RBD devices, and quite possibly list them as being "available". This causes a couple of problems:

1) Because /dev/rbd0 appears in the list of available devices, the orchestrator will actually try to deploy OSDs on top of those RBD devices. Luckily, this will fail, because the various LVM invocations will die with "Device /dev/rbd0 excluded by a filter", but really we shouldn't even be trying to do this in the first place. Let's not rely on luck ;-)
2) It's possible for /dev/rbd* devices to be locked/stuck in such a way that when ceph-volume invokes blkid, it hangs indefinitely (the process ends up in D-state). This can actually block the entire orchestrator, because the orchestrator calls out to cephadm periodically to inventory devices, and the latter tries to acquire a lock, which it can't get because a prior invocation is stuck running ceph-volume inventory.

I suggest we make ceph-volume completely ignore /dev/rbd* when doing a device inventory. I know we had a similar discussion on dev@ceph.io regarding ceph-volume listing, or not listing, GPT devices (see https://lists.ceph.io/hyperkitty/list/dev@ceph.io/thread/N3TK4IO2QYHXIZMQTZ4AMPU5BE56J5MP/#T7UM53WCW2MDD62DDH6KLI4EZXKBXZBY) but the difference here is that mapped RBD volumes really aren't part of the host inventory, so IMO should be excluded.