[rbd-mirror] entries_behind_master will not be zero after mirror over
I have two ceph-12.2.4 cluster. Rbd mirror deamon run on cluster two. poolclz/rbdclz on cluster one is primary image and non-primary on cluster two.
When I write data in to poolclz/rbdclz by using fio or rbd bench, rbd mirror will data to cluster.
I find the entries_behind_master can't decrease to zero by using command "rbd mirror image status poolclz/rbdclz". That mean mirror_position is some entries behind master_position.I think this is unacceptable.
rbdclz: global_id: 105d7a59-06c0-404d-878e-80dadba39c55 state: up+replaying description: replaying, master_position=[object_number=7, tag_tid=30, entry_tid=295], mirror_position=[object_number=7, tag_tid=30, entry_tid=287], entries_behind_master=8 last_update: 2018-03-23 20:09:39
#1 Updated by Jason Dillaman 10 months ago
- Priority changed from Normal to Low
This is actually just an artifact of IO batching within rbd-mirror. The commit position is only updated after every 32 IOs in the absence of a flush request. Under a real workload, you would have flush requests so the updated commit position would be appropriately advanced. If you were to take a snapshot of your source image and compare it to your destination, you would discover that both images are identical (you would also discover your 'entries_behind_master' dropping to zero since a snapsoht would flush).
#2 Updated by liuzhong chen 10 months ago
Thank you！ @Jason Dillaman
I have anther two question.I find the non-primary image used volumes is equal to primary image used volumes although the entries_behind_master is not zero.It means just the mirror_commit_position not update but all the date have been mirror to mirror image.Am I right?
[root@node-1 ~]# rbd du poolclz1/rbdclz1 NAME PROVISIONED USED rbdclz1 1024M 940M
[root@node-1 ~]# ssh node-2 Last login: Fri Mar 30 01:51:34 2018 from 192.168.122.1 [root@node-2 ~]# rbd mirror image status poolclz1/rbdclz1 rbdclz1: global_id: 09febb93-3eac-4467-a284-f02f90b846b5 state: up+replaying description: replaying, master_position=[object_number=3, tag_tid=2, entry_tid=619], mirror_position=[object_number=3, tag_tid=2, entry_tid=607], entries_behind_master=12 last_update: 2018-03-30 02:08:19
[root@node-2 ~]# rbd du poolclz1/rbdclz1 NAME PROVISIONED USED rbdclz1 1024M 940M
#3 Updated by liuzhong chen 10 months ago
Another question is if I use fio to write for 60s.Is there any other parameter to indicate that the mirror is over?
If there is no other paramete,I think if the entries_behind_master is less than 32,I can affirm the mirror is over.Whether this method is feasible or not.
#6 Updated by Jason Dillaman 8 months ago
- Subject changed from rbd mirror: entries_behind_master will not be zero after mirror over to [rbd-mirror] entries_behind_master will not be zero after mirror over
- Status changed from New to In Progress
- Assignee set to Jason Dillaman
- Backport set to jewel,luminous,mimic