Ceph » rgw

• We have one RGW on each Site and RGW multi-site replication is set between them. Data added in primary gets replicated to secondary.
• Primary side is read-write-delete enabled while secondary side is only read-only.
• If we make secondary RGW service down for some minutes and keep adding data at the primary side while secondary service is down, following happens :
  • Data gets added into the primary side
  • When secondary service comes up, it does not get the data from primary side which was added while secondary service was down
  • If we add New data in primary(after secondary coming up), then this time only newly added data gets synced from primary to secondary and we get a sync status showing "Data is caught up with the source"
  • In this case, data added on primary while secondary being down, does not get written on secondary side after it comes up

Does anyone know why this happen? What could be the solution to this problem?

What is the full-proof way to tell that secondary side has really caught up with the master? Thanks in advance!