Transparently support migrating images with minimal/zero downtime
Two use-cases here:
(1) Need to support transitioning legacy v1 images to new v2 images. An updated librbd client should be able to detect a special metadata flag against the v1 image stating it should be migrated to a new v2 image.
(2) Need to support migrating images between pools. Same concept where the special metadata flag would include a metadata redirect to where the image should be located.
The new redirected image destination would have a feature flag indicating that a migration is in-progress. Similar to how clones operate, images with the migration feature flag would first try IO against themselves and re-direct to the "parent" if the backing object doesn't exist. Upon write, a full history copy-up would be performed (i.e. include the full snapshot state but translated for the new destination image).
The rbd CLI would receive new actions to start an image migration, "flatten" a migrating image, and to finalize the migration (e.g. delete the original image and unset the migration feature flag).
In order for a live client to detect the new migration destination, the image would need to be re-opened. This could be done via a QEMU live-migration or a process restart after using the rbd CLI to start the migration on an image.
After this feature has been in two LTS releases, we can remove support for v1 images since that will provide a supported upgrade path where all v1 images could be migrated to v2. Once support for v1 images are removed, the OSDs can remove support for tmap.