Ceph » rgw

When a metadata request (such as bucket a operation) is sent to a non-master zone, the request is forwarded to the metadata master zone. On a successful reply from the master, we return that to the client. But this only guarantees that the metadata update has been applied on the master - it's not applied on the non-master zone until metadata sync fetches the update. This leaves a window between responding to the client and fetching the updated metadata where the changes are not visible to clients of the non-master zone.

The issue came up in http://tracker.ceph.com/issues/18003 with respect to bucket versioning changes, and was fixed by writing the change locally after getting a successful reply from the master. This could lead to races between the write and other metadata updates synced from the master. It could also lead to inconsistencies in the metadata itself, like the mtime for example.

The non-master zone should probably just read the updated metadata from the master zone, and write that locally. This should work as long as this write and the writes from metadata sync both respect the same object versioning requirements.