Bug #17993
closed
rbd-mirror: potential race mirroring cloned image
Added by Jason Dillaman over 7 years ago.
Updated over 5 years ago.
Description
With pool-level replication enabled, when a clone is created, it first creates an empty standalone image and adds it to the mirroring directory. If the remote rbd-mirror daemon detects the new image before the clone process completes, it the replication will fail since it will try to create a non-cloned image on the remote side. This is a very small race potential that was discovered after improving new replicated image discovery.
Hi Jason,
What about add a key-value in header object to mark this image is ready for mirror or not? such as "mirror_ready = true|false".
Then when we are cloning an image, put an false in mirror_ready until all work finished, when rbd-mirror detect a new image in this pool, rbd-mirror should
check the mirror_ready at first, if false, wait a moment and check mirror_ready again until it's ready or timed out.
Does this sound good?
Jason Dillaman wrote:
With pool-level replication enabled, when a clone is created, it first creates an empty standalone image and adds it to the mirroring directory. If the remote rbd-mirror daemon detects the new image before the clone process completes, it the replication will fail since it will try to create a non-cloned image on the remote side. This is a very small race potential that was discovered after improving new replicated image discovery.
We track this state already -- it's just being enabled too soon since clone is re-using the create image state machine. Therefore, the create image state machine just needs a bool to say "don't enable mirroring yet" and then instead the clone state machine will enable mirroring once it has completed setting the parent.
Jason Dillaman wrote:
We track this state already -- it's just being enabled too soon since clone is re-using the create image state machine. Therefore, the create image state machine just needs a bool to say "don't enable mirroring yet" and then instead the clone state machine will enable mirroring once it has completed setting the parent.
Oh, yes, it's in RBD_MIRROR object. Okey, will reuse it. thanx
Yang Dongsheng wrote:
Jason Dillaman wrote:
We track this state already -- it's just being enabled too soon since clone is re-using the create image state machine. Therefore, the create image state machine just needs a bool to say "don't enable mirroring yet" and then instead the clone state machine will enable mirroring once it has completed setting the parent.
Oh, yes, it's in RBD_MIRROR object. Okey, will reuse it. thanx
As I have a blocked pull request about CloneRequest, I will put this after CloneRequest merged.
- Status changed from New to In Progress
- Assignee set to Jason Dillaman
- Status changed from In Progress to Fix Under Review
- Status changed from Fix Under Review to Pending Backport
- Copied to Backport #18500: jewel: rbd-mirror: potential race mirroring cloned image added
- Copied to Backport #18501: kraken: rbd-mirror: potential race mirroring cloned image added
I wonder this patch was signed backport to Jewel but not do it.Is there some problem to backport to jewel or something else?Thank you!
I attempted the jewel backport at #18500 but it is beyond my abilities.
Nathan Cutler wrote:
I attempted the jewel backport at #18500 but it is beyond my abilities.
@Nathan Weinberg OK,thanks.
- Status changed from Pending Backport to Resolved
Also available in: Atom
PDF