Project

General

Profile

Bug #17993

rbd-mirror: potential race mirroring cloned image

Added by Jason Dillaman 4 months ago. Updated 2 months ago.

Status:
Pending Backport
Priority:
Low
Target version:
-
Start date:
11/22/2016
Due date:
% Done:

0%

Source:
Tags:
Backport:
jewel, kraken
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Release:
Needs Doc:
No

Description

With pool-level replication enabled, when a clone is created, it first creates an empty standalone image and adds it to the mirroring directory. If the remote rbd-mirror daemon detects the new image before the clone process completes, it the replication will fail since it will try to create a non-cloned image on the remote side. This is a very small race potential that was discovered after improving new replicated image discovery.


Related issues

Copied to Backport #18500: jewel: rbd-mirror: potential race mirroring cloned image Need More Info
Copied to Backport #18501: kraken: rbd-mirror: potential race mirroring cloned image New

History

#1 Updated by Yang Dongsheng 4 months ago

Hi Jason,
What about add a key-value in header object to mark this image is ready for mirror or not? such as "mirror_ready = true|false".

Then when we are cloning an image, put an false in mirror_ready until all work finished, when rbd-mirror detect a new image in this pool, rbd-mirror should
check the mirror_ready at first, if false, wait a moment and check mirror_ready again until it's ready or timed out.

Does this sound good?

Jason Dillaman wrote:

With pool-level replication enabled, when a clone is created, it first creates an empty standalone image and adds it to the mirroring directory. If the remote rbd-mirror daemon detects the new image before the clone process completes, it the replication will fail since it will try to create a non-cloned image on the remote side. This is a very small race potential that was discovered after improving new replicated image discovery.

#2 Updated by Jason Dillaman 4 months ago

We track this state already -- it's just being enabled too soon since clone is re-using the create image state machine. Therefore, the create image state machine just needs a bool to say "don't enable mirroring yet" and then instead the clone state machine will enable mirroring once it has completed setting the parent.

#3 Updated by Yang Dongsheng 4 months ago

Jason Dillaman wrote:

We track this state already -- it's just being enabled too soon since clone is re-using the create image state machine. Therefore, the create image state machine just needs a bool to say "don't enable mirroring yet" and then instead the clone state machine will enable mirroring once it has completed setting the parent.

Oh, yes, it's in RBD_MIRROR object. Okey, will reuse it. thanx

#4 Updated by Yang Dongsheng 4 months ago

Yang Dongsheng wrote:

Jason Dillaman wrote:

We track this state already -- it's just being enabled too soon since clone is re-using the create image state machine. Therefore, the create image state machine just needs a bool to say "don't enable mirroring yet" and then instead the clone state machine will enable mirroring once it has completed setting the parent.

Oh, yes, it's in RBD_MIRROR object. Okey, will reuse it. thanx

As I have a blocked pull request about CloneRequest, I will put this after CloneRequest merged.

#5 Updated by Jason Dillaman 4 months ago

  • Status changed from New to In Progress
  • Assignee set to Jason Dillaman

#6 Updated by Jason Dillaman 4 months ago

  • Status changed from In Progress to Need Review

#7 Updated by Mykola Golub 2 months ago

  • Status changed from Need Review to Pending Backport

#8 Updated by Loic Dachary 2 months ago

  • Copied to Backport #18500: jewel: rbd-mirror: potential race mirroring cloned image added

#9 Updated by Loic Dachary 2 months ago

  • Copied to Backport #18501: kraken: rbd-mirror: potential race mirroring cloned image added

Also available in: Atom PDF