Bug #58887
closed[rbd-mirror] don't leave an incomplete primary snapshot if the peer who is handling snapshot creation dies
0%
Description
At a high level, creating a primary snapshot consists of three steps:
1. actually creating a snapshot in the mirror namespace (initially the snapshot is marked incomplete so that rbd-mirror daemon can skip it if it gets to it too early)
2. generating a set of image state objects with additional metadata for the snapshot
3. marking the snapshot as complete after the image state objects are written out
Depending on the circumstances, a request to create a primary snapshot can be forwarded to rbd-mirror daemon. If that happens and rbd-mirror daemon gets axed for some practical reason after completing steps (1) and/or (2) but before completing step (3), we are left with a permanently incomplete primary snapshot because upon retrying that primary snapshot creation request, librbd notices that such snapshot already exists. It does not check whether this "pre-existing" snapshot is complete.
we indeed have an incomplete remote snapshot (ID -- 79601 for the first image). Here is the relevant excerpt for the first image:
2022-08-08T15:05:05.346+0000 7f03c6f28700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 handle_image_update_notify:
2022-08-08T15:05:05.346+0000 7f03c6f28700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 handle_image_update_notify: restarting idle replayer
2022-08-08T15:05:05.346+0000 7f03c6f28700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 load_local_image_meta:
2022-08-08T15:05:05.347+0000 7f03d5745700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 handle_load_local_image_meta: r=-2
2022-08-08T15:05:05.347+0000 7f03d5745700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 refresh_remote_image:
2022-08-08T15:05:06.132+0000 7f03c6f28700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 handle_image_update_notify:
2022-08-08T15:05:06.132+0000 7f03c6f28700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 handle_image_update_notify: flagging snapshot rescan required
2022-08-08T15:05:12.658+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 handle_refresh_remote_image: r=0
2022-08-08T15:05:12.658+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_local_mirror_snapshots:
2022-08-08T15:05:12.658+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_local_mirror_snapshots: local mirror snapshot: id=38890, mirror_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=601da92e-6085-44c3-ba19-055e1ba820a5, primary_snap_id=1365d, last_copied_object_number=33536, snap_seqs={79453=18446744073709551614}]
2022-08-08T15:05:12.658+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_local_mirror_snapshots: found local mirror snapshot: local_snap_id_start=38890, local_snap_id_end=18446744073709551614, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=601da92e-6085-44c3-ba19-055e1ba820a5, primary_snap_id=1365d, last_copied_object_number=33536, snap_seqs={79453=18446744073709551614}]
2022-08-08T15:05:12.658+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots:
2022-08-08T15:05:12.658+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=79453, mirror_ns=[mirror state=primary, complete=1, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:05:12.658+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: skipping synced remote snapshot 79453
2022-08-08T15:05:12.658+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=79601, mirror_ns=[mirror state=primary, complete=0, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:05:12.658+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=80109, mirror_ns=[mirror state=primary, complete=1, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:05:12.658+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=80518, mirror_ns=[mirror state=primary, complete=1, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:05:12.658+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=160115, mirror_ns=[mirror state=primary, complete=1, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:05:12.658+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: found remote mirror snapshot: remote_snap_id_start=79453, remote_snap_id_end=79601, remote_snap_ns=[mirror state=primary, complete=0, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:05:12.658+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot not complete
2022-08-08T15:05:12.658+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: restarting snapshot scan due to remote update notification
2022-08-08T15:05:12.658+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 load_local_image_meta:
2022-08-08T15:05:12.659+0000 7f03d5745700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 handle_load_local_image_meta: r=-2
2022-08-08T15:05:12.659+0000 7f03d5745700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 refresh_remote_image:
2022-08-08T15:05:14.103+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 handle_refresh_remote_image: r=0
2022-08-08T15:05:14.103+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_local_mirror_snapshots:
2022-08-08T15:05:14.103+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_local_mirror_snapshots: local mirror snapshot: id=38890, mirror_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=601da92e-6085-44c3-ba19-055e1ba820a5, primary_snap_id=1365d, last_copied_object_number=33536, snap_seqs={79453=18446744073709551614}]
2022-08-08T15:05:14.103+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_local_mirror_snapshots: found local mirror snapshot: local_snap_id_start=38890, local_snap_id_end=18446744073709551614, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=601da92e-6085-44c3-ba19-055e1ba820a5, primary_snap_id=1365d, last_copied_object_number=33536, snap_seqs={79453=18446744073709551614}]
2022-08-08T15:05:14.103+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots:
2022-08-08T15:05:14.103+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=79453, mirror_ns=[mirror state=primary, complete=1, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:05:14.103+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: skipping synced remote snapshot 79453
2022-08-08T15:05:14.103+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=79601, mirror_ns=[mirror state=primary, complete=0, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:05:14.103+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=80109, mirror_ns=[mirror state=primary, complete=1, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:05:14.103+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=80518, mirror_ns=[mirror state=primary, complete=1, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:05:14.103+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=160115, mirror_ns=[mirror state=primary, complete=1, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:05:14.103+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: found remote mirror snapshot: remote_snap_id_start=79453, remote_snap_id_end=79601, remote_snap_ns=[mirror state=primary, complete=0, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:05:14.103+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot not complete
2022-08-08T15:05:14.103+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: all remote snapshots synced: idling waiting for new snapshot
2022-08-08T15:05:14.103+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 notify_status_updated:
2022-08-08T15:05:14.103+0000 7f03dbf52700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 get_replay_status:
2022-08-08T15:05:14.103+0000 7f03dbf52700 15 rbd::mirror::image_replayer::snapshot::util::compute_remote_snap_id: local snapshot 38890 maps to remote snapshot 1365d
2022-08-08T15:05:14.103+0000 7f03dbf52700 15 rbd::mirror::ImageReplayer: 0x559593076f00 [1/d58a49cb-b2cb-4181-b4de-ffffbd9b8f12] operator(): replay status ready: r=-17
2022-08-08T15:05:14.103+0000 7f03dbf52700 15 rbd::mirror::ImageReplayer: 0x559593076f00 [1/d58a49cb-b2cb-4181-b4de-ffffbd9b8f12] set_mirror_image_status_update: status={state=up+replaying, description=replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"local_snapshot_timestamp":1659825349,"remote_snapshot_ti
mestamp":1659971104,"replay_state":"syncing","syncing_percent":3353600,"syncing_snapshot_timestamp":1659971104}, last_update=0.000000]}
... and rbd-mirror daemon goes on to wait for the next mirror snapshot to be created. When that happens, it again encounters the same incomplete snapshot and everything repeats:
2022-08-08T15:10:29.801+0000 7f03c6f28700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 handle_image_update_notify:
2022-08-08T15:10:29.801+0000 7f03c6f28700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 handle_image_update_notify: restarting idle replayer
2022-08-08T15:10:29.801+0000 7f03c6f28700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 load_local_image_meta:
2022-08-08T15:10:29.802+0000 7f03d5745700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 handle_load_local_image_meta: r=-2
2022-08-08T15:10:29.802+0000 7f03d5745700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 refresh_remote_image:
2022-08-08T15:10:30.512+0000 7f03c6f28700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 handle_image_update_notify:
2022-08-08T15:10:30.512+0000 7f03c6f28700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 handle_image_update_notify: flagging snapshot rescan required
2022-08-08T15:10:32.726+0000 7f03ce737700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 handle_refresh_remote_image: r=0
2022-08-08T15:10:32.726+0000 7f03ce737700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_local_mirror_snapshots:
2022-08-08T15:10:32.726+0000 7f03ce737700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_local_mirror_snapshots: local mirror snapshot: id=38890, mirror_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=601da92e-6085-44c3-ba19-055e1ba820a5, primary_snap_id=1365d, last_copied_object_number=33536, snap_seqs={79453=18446744073709551614}]
2022-08-08T15:10:32.726+0000 7f03ce737700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_local_mirror_snapshots: found local mirror snapshot: local_snap_id_start=38890, local_snap_id_end=18446744073709551614, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=601da92e-6085-44c3-ba19-055e1ba820a5, primary_snap_id=1365d, last_copied_object_number=33536, snap_seqs={79453=18446744073709551614}]
2022-08-08T15:10:32.726+0000 7f03ce737700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots:
2022-08-08T15:10:32.726+0000 7f03ce737700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=79453, mirror_ns=[mirror state=primary, complete=1, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:10:32.726+0000 7f03ce737700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: skipping synced remote snapshot 79453
2022-08-08T15:10:32.726+0000 7f03ce737700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=79601, mirror_ns=[mirror state=primary, complete=0, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:10:32.726+0000 7f03ce737700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=80109, mirror_ns=[mirror state=primary, complete=1, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:10:32.726+0000 7f03ce737700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=80518, mirror_ns=[mirror state=primary, complete=1, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:10:32.726+0000 7f03ce737700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=160272, mirror_ns=[mirror state=primary, complete=1, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:10:32.726+0000 7f03ce737700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: found remote mirror snapshot: remote_snap_id_start=79453, remote_snap_id_end=79601, remote_snap_ns=[mirror state=primary, complete=0, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:10:32.726+0000 7f03ce737700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot not complete
2022-08-08T15:10:32.726+0000 7f03ce737700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: restarting snapshot scan due to remote update notification
2022-08-08T15:10:32.726+0000 7f03ce737700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 load_local_image_meta:
2022-08-08T15:10:32.726+0000 7f03d5745700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 handle_load_local_image_meta: r=-2
2022-08-08T15:10:32.726+0000 7f03d5745700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 refresh_remote_image:
2022-08-08T15:10:34.242+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 handle_refresh_remote_image: r=0
2022-08-08T15:10:34.242+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_local_mirror_snapshots:
2022-08-08T15:10:34.242+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_local_mirror_snapshots: local mirror snapshot: id=38890, mirror_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=601da92e-6085-44c3-ba19-055e1ba820a5, primary_snap_id=1365d, last_copied_object_number=33536, snap_seqs={79453=18446744073709551614}]
2022-08-08T15:10:34.242+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_local_mirror_snapshots: found local mirror snapshot: local_snap_id_start=38890, local_snap_id_end=18446744073709551614, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=601da92e-6085-44c3-ba19-055e1ba820a5, primary_snap_id=1365d, last_copied_object_number=33536, snap_seqs={79453=18446744073709551614}]
2022-08-08T15:10:34.242+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots:
2022-08-08T15:10:34.242+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=79453, mirror_ns=[mirror state=primary, complete=1, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:10:34.242+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: skipping synced remote snapshot 79453
2022-08-08T15:10:34.242+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=79601, mirror_ns=[mirror state=primary, complete=0, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:10:34.242+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=80109, mirror_ns=[mirror state=primary, complete=1, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:10:34.242+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=80518, mirror_ns=[mirror state=primary, complete=1, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:10:34.242+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot: id=160272, mirror_ns=[mirror state=primary, complete=1, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:10:34.242+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: found remote mirror snapshot: remote_snap_id_start=79453, remote_snap_id_end=79601, remote_snap_ns=[mirror state=primary, complete=0, mirror_peer_uuids=6b834d11-7ff8-4b12-8c2d-0980f038d160, clean_since_snap_id=head]
2022-08-08T15:10:34.242+0000 7f03cef38700 15 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: remote mirror snapshot not complete
2022-08-08T15:10:34.242+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 scan_remote_mirror_snapshots: all remote snapshots synced: idling waiting for new snapshot
2022-08-08T15:10:34.242+0000 7f03cef38700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 notify_status_updated:
2022-08-08T15:10:34.242+0000 7f03dbf52700 10 rbd::mirror::image_replayer::snapshot::Replayer: 0x559592b2e800 get_replay_status:
2022-08-08T15:10:34.242+0000 7f03dbf52700 15 rbd::mirror::image_replayer::snapshot::util::compute_remote_snap_id: local snapshot 38890 maps to remote snapshot 1365d
2022-08-08T15:10:34.243+0000 7f03dbf52700 15 rbd::mirror::ImageReplayer: 0x559593076f00 [1/d58a49cb-b2cb-4181-b4de-ffffbd9b8f12] operator(): replay status ready: r=-17
2022-08-08T15:10:34.243+0000 7f03dbf52700 15 rbd::mirror::ImageReplayer: 0x559593076f00 [1/d58a49cb-b2cb-4181-b4de-ffffbd9b8f12] set_mirror_image_status_update: status={state=up+replaying, description=replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"local_snapshot_timestamp":1659825349,"remote_snapshot_timestamp":1659971429,"replay_state":"syncing","syncing_percent":3353600,"syncing_snapshot_timestamp":1659971429}, last_update=0.000000]}
Updated by Ilya Dryomov about 1 year ago
- Status changed from New to Fix Under Review
- Pull request ID set to 50324
Updated by Ilya Dryomov about 1 year ago
- Status changed from Fix Under Review to Pending Backport
Updated by Backport Bot about 1 year ago
- Copied to Backport #59506: pacific: [rbd-mirror] don't leave an incomplete primary snapshot if the peer who is handling snapshot creation dies added
Updated by Backport Bot about 1 year ago
- Copied to Backport #59507: quincy: [rbd-mirror] don't leave an incomplete primary snapshot if the peer who is handling snapshot creation dies added
Updated by Backport Bot 11 months ago
- Status changed from Pending Backport to Resolved
While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".