Feature #11881
Updated by Loïc Dachary almost 9 years ago
See https://github.com/ceph/ceph/pull/1229
* ceph-disk must understand the underlying infrastructure (devmapper instead of /dev/XXX)
* ceph-disk must not be confused by the fact that two paths point to the same device
* the partitions must be created on the actual hardware, the devmapper names can't be used
* use real hardware to figure things out (find one)
* write tests using isci multipath
* DM_MULTIPATH_DEVICE_PATH udev environment variable. This is set by 62-multipath.rules for all devices that will become paths in a multipath device, and all their partitions. There are a couple gotchas, however.
** It won't get set the very first time a piece of storage hardware is seen by the system. With find_multipaths enabled, there's no way for multipath to know ahead of time if a new piece of storage will be multipathable when it first appears.
** If the storage gets discovered in the initramfs, multipath won't be able to update it's list of multipathable devices there. This means it will fail to get correctly labelled in the initramfs every time, until the initramfs is remade. That's why it is advisable to remake the initramfs after adding new hardware to a multipath system (there are ways of working around this if that's incredibly important).
But aside from new hardware (which probably won't be labelled for
ceph-disk the first time it appears anyway), this should work to keep
you from messing with multipath's path devices.
Simple way to set up multipath without actually having any multipath hardware:
* modprobe scsi_debug vpd_use_hostno=0 add_host=2 dev_size_mb=100
will get you 2 100MB scsi_debug devices that multipath will happily set
itself up on. But I don't know if this will help you, since you can't
choose what's on the device when it gets discovered. The device usually
(always?) comes up with the same WWID, so after the first time, the
multipath udev rules will flag it correctly, but I doubt that the
95-ceph-osd.rules would pick it up.
Probably a better way, is to simply pick an unused scsi device,
and run
* multipath <scsi_device>
Multipath will make a multipath device with only one path on top of it,
and will remember that it is supposed to be multipathed in the future.
Once you're all one with it, run:
* multipath -f <multipath_device>
to remove the multipath device, and
* multipath -w <scsi_device>
To wipe the information that it's supposed to be a multipath device (so
that it won't automatically get multipathed the next time you start up).