Ceph » rbd

$ rbd create --size 1G img
$ rbd snap create img@snap
Creating snap: 100% complete...done.
$ rbd snap protect img@snap
$ rbd clone img@snap cloneimg
$ rbd migration prepare cloneimg migr-cloneimg --flatten
$ rbd info migr-cloneimg
rbd image 'migr-cloneimg':
        size 1 GiB in 256 objects
        order 22 (4 MiB objects)
        snapshot_count: 0
        id: 104a257f97c2
        block_name_prefix: rbd_data.104a257f97c2
        format: 2
        features: layering, exclusive-lock, object-map, fast-diff, deep-flatten, migrating
        op_features:
        flags:
        create_timestamp: Wed May  1 12:52:52 2024
        access_timestamp: Wed May  1 12:52:52 2024
        modify_timestamp: Wed May  1 12:52:52 2024
        parent: rbd/img@snap
        overlap: 1 GiB
$ rbd children img@snap                                                                                                                            
rbd/migr-cloneimg
$ rbd migration execute migr-cloneimg
Image migration: 100% complete...done.
$ rbd migration commit migr-cloneimg
Commit image migration: 100% complete...done.

At this point migr-cloneimg is expected to be fully detached from the parent, on both sides. This is the case for migr-cloneimg side, but not for img@snap side:

$ rbd info migr-cloneimg
rbd image 'migr-cloneimg':
        size 1 GiB in 256 objects
        order 22 (4 MiB objects)
        snapshot_count: 0
        id: 104a257f97c2
        block_name_prefix: rbd_data.104a257f97c2
        format: 2
        features: layering, exclusive-lock, object-map, fast-diff, deep-flatten
        op_features:
        flags:
        create_timestamp: Wed May  1 12:52:52 2024
        access_timestamp: Wed May  1 12:52:52 2024
        modify_timestamp: Wed May  1 12:52:52 2024
$ rbd children img@snap
rbd/migr-cloneimg

Since there is still a record of a clone for img@snap, we can't unprotect it -- not even after migr-cloneimg is removed:

$ rbd snap unprotect img@snap                                                                                                                      
2024-05-01T12:53:42.268+0200 7f14277fe700 -1 librbd::SnapshotUnprotectRequest: cannot unprotect: at least 1 child(ren) [104a257f97c2] in pool 'rbd'
2024-05-01T12:53:42.268+0200 7f14277fe700 -1 librbd::SnapshotUnprotectRequest: encountered error: (16) Device or resource busy
2024-05-01T12:53:42.268+0200 7f14277fe700 -1 librbd::SnapshotUnprotectRequest: 0x556ca7920ec0 should_complete_error: ret_val=-16
2024-05-01T12:53:42.276+0200 7f1426ffd700 -1 librbd::SnapshotUnprotectRequest: 0x556ca7920ec0 should_complete_error: ret_val=-16
rbd: unprotecting snap failed: (16) Device or resource busy
$ rbd rm migr-cloneimg
Removing image: 100% complete...done.
$ rbd snap unprotect img@snap                                                                                                                      
2024-05-01T12:54:37.503+0200 7f2a29ac6700 -1 librbd::SnapshotUnprotectRequest: cannot unprotect: at least 1 child(ren) [104a257f97c2] in pool 'rbd'
2024-05-01T12:54:37.503+0200 7f2a29ac6700 -1 librbd::SnapshotUnprotectRequest: encountered error: (16) Device or resource busy
2024-05-01T12:54:37.503+0200 7f2a29ac6700 -1 librbd::SnapshotUnprotectRequest: 0x562c0bff0ec0 should_complete_error: ret_val=-16
2024-05-01T12:54:37.507+0200 7f2a292c5700 -1 librbd::SnapshotUnprotectRequest: 0x562c0bff0ec0 should_complete_error: ret_val=-16
rbd: unprotecting snap failed: (16) Device or resource busy 

The only thing that changes with migr-cloneimg removal is "rbd children" starts throwing an error:

$ rbd children img@snap                                                                                                                            
rbd: listing children failed: (2) No such file or directory                               
2024-05-01T12:54:33.691+0200 7fc39edec740 -1 librbd::api::Image: list_descendants: error looking up name for image id 104a257f97c2 in pool rbd

This extends to clone v2 clones as well -- since unprotecting isn't needed there, img@snap gets moved to the trash but is never removed from the trash.