Bug #63178
closedmultisite: don't write data/bilog entries for lifecycle transitions/deletes
0%
Description
lifecycle processing should happen independently on all zones, so the lifecycle deletes and transitions probably don't need to be replicated. heavy lifecycle activity can generate a lot of replication logs and slow down multisite replication significantly
rgw::sal::Object::transition() and rgw::sal::Object::DeleteOp should be extended to take a bool log_op to control whether replication logs need to be written. the calls from lifecycle can all pass false, but other calls should reproduce the existing checks for need_to_log_data() that currently happen at lower levels in rgw_rados.cc
Updated by Matt Benjamin 7 months ago
sounds good to me; I think we'd need some way to describe the intent of the delete in the rgw::sal::Object::DeleteOp?
Matt
Updated by Casey Bodley 7 months ago
with versioned buckets, there may be cases where lifecycle would generate different version ids in different zones - either from transitions or creation of delete markers. i'm not sure exactly how this works currently, but it could be a potential regression
Updated by Casey Bodley 7 months ago
this would also cause issues for 'sync modules' like elasticsearch and cloud sync, who don't themselves run lifecycle processing so wouldn't be informed of deletion/transition events. but it's hard to prioritize support for these things over general performance and reliability of multisite replication
Updated by Casey Bodley 4 months ago
- Status changed from Fix Under Review to Pending Backport
- Tags changed from multisite to multisite lifecycle
Updated by Backport Bot 4 months ago
- Copied to Backport #64088: reef: multisite: don't write data/bilog entries for lifecycle transitions/deletes added
Updated by Backport Bot 4 months ago
- Tags changed from multisite lifecycle to multisite lifecycle backport_processed
Updated by Jane Zhu 3 months ago
Missing some places, in the PR https://github.com/ceph/ceph/pull/54759, when converting the boolean argument to "uint32_t flags" in the callers of the function complete(...).
A fixup PR: https://github.com/ceph/ceph/pull/55293