Project

General

Profile

Actions

Bug #16473

closed

Demotion/promotion results in the image syncing again from the beginning

Added by Jason Dillaman almost 8 years ago. Updated over 7 years ago.

Status:
Resolved
Priority:
High
Assignee:
Jason Dillaman
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
jewel
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Demote the primary image and promote the original non-primary image. The rbd-mirror for the newly demoted non-primary image will incorrectly start a full image sync instead of just replaying the journal deltas:

2016-06-24 20:46:47.607020 7fafb97fa700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 handle_get_local_image_id: r=0
2016-06-24 20:46:47.607041 7fafb97fa700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 get_remote_tag_class
2016-06-24 20:46:47.607045 7fafb97fa700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 update_progress: GET_REMOTE_TAG_CLASS
2016-06-24 20:46:47.902096 7faf8e7fc700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 handle_get_remote_tag_class: r=0
2016-06-24 20:46:47.902099 7faf8e7fc700 10 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 handle_get_remote_tag_class: remote tag class=0
2016-06-24 20:46:47.902099 7faf8e7fc700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 get_client
2016-06-24 20:46:47.902100 7faf8e7fc700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 update_progress: GET_CLIENT
2016-06-24 20:46:48.487467 7faf8e7fc700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 handle_get_client: r=0
2016-06-24 20:46:48.487479 7faf8e7fc700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 decode_client_meta
2016-06-24 20:46:48.487484 7faf8e7fc700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 decode_client_meta: client found: image_id=11d54238e1f29, client_meta=[image_id=11d54238e1f29, state=Syncing, sync_object_count=0, sync_points=[], snap_seqs=[]

                               ^^^^^^^ -- state should be MIRROR_PEER_STATE_REPLAYING, not MIRROR_PEER_STATE_SYNCING

2016-06-24 20:46:48.487488 7faf8e7fc700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 open_remote_image
2016-06-24 20:46:48.487491 7faf8e7fc700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 update_progress: OPEN_REMOTE_IMAGE
2016-06-24 20:48:03.877278 7faf8cbf5700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 handle_open_remote_image: r=0
2016-06-24 20:48:05.676594 7faf8cbf5700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 open_local_image
2016-06-24 20:48:05.676605 7faf8cbf5700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 update_progress: OPEN_LOCAL_IMAGE
2016-06-24 20:48:06.158133 7faf268f6700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 handle_open_local_image: r=0
2016-06-24 20:48:06.158134 7faf268f6700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 update_client
2016-06-24 20:48:06.158135 7faf268f6700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 update_progress: UPDATE_CLIENT
2016-06-24 20:48:06.158146 7faf268f6700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 get_remote_tags
2016-06-24 20:48:06.158147 7faf268f6700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 update_progress: GET_REMOTE_TAGS
2016-06-24 20:48:06.158150 7faf268f6700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 get_remote_tags
2016-06-24 20:48:06.451701 7faf8e7fc700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 handle_get_remote_tags: r=0
2016-06-24 20:48:06.451716 7faf8e7fc700 10 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 handle_get_remote_tags: decoded remote tag: [mirror_uuid=9b6fa8da-3001-4d3a-ac18-697b8318f3ce, predecessor_mirror_uuid=]
2016-06-24 20:48:06.451721 7faf8e7fc700 10 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 handle_get_remote_tags: decoded remote tag: [mirror_uuid=9b6fa8da-3001-4d3a-ac18-697b8318f3ce, predecessor_mirror_uuid=9b6fa8da-3001-4d3a-ac18-697b8318f3ce, predecessor_tag_tid=11, predecessor_entry_tid=61272]
2016-06-24 20:48:06.451727 7faf8e7fc700 10 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 handle_get_remote_tags: decoded remote tag: [mirror_uuid=<orphan>, predecessor_mirror_uuid=9b6fa8da-3001-4d3a-ac18-697b8318f3ce, predecessor_tag_tid=12, predecessor_entry_tid=36586]
2016-06-24 20:48:06.451734 7faf8e7fc700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 handle_get_remote_tags: local tag data: [mirror_uuid=<orphan>, predecessor_mirror_uuid=, predecessor_tag_tid=12, predecessor_entry_tid=36586]
2016-06-24 20:48:06.451738 7faf8e7fc700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 handle_get_remote_tags: local image was demoted

** correctly detects that the local image was demoted, but ...

2016-06-24 20:48:06.451741 7faf8e7fc700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 image_sync

*** ... incorrectly starts the image-sync instead of starting the replay due to the wrong state above

2016-06-24 20:48:06.451743 7faf8e7fc700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 update_progress: IMAGE_SYNC
2016-06-24 20:48:06.451755 7faf8e7fc700 20 rbd::mirror::image_replayer::BootstrapRequest: 0x7faf641497c0 image_sync
Actions #1

Updated by Jason Dillaman almost 8 years ago

  • Status changed from New to In Progress
  • Assignee set to Jason Dillaman
  • Backport set to jewel
Actions #2

Updated by Ken Dreyer over 7 years ago

Jason, is this fixed by https://github.com/ceph/ceph/pull/10703 ?

Actions #3

Updated by Jason Dillaman over 7 years ago

@Ken: most likely, yes -- but there isn't an explicit test case to verify that it is properly functioning.

Actions #4

Updated by Jason Dillaman over 7 years ago

  • Status changed from In Progress to Resolved
Actions

Also available in: Atom PDF