Bug #62075
closedNew radosgw-admin commands to cleanup leftover OLH index entries and unlinked instance objects
100%
Description
The transactions that are associated with object creation and deletion within versioned buckets involve OLH objects, instance objects, and several types of index entries. It is intrinsically possible that one or more of these resources is left in an inconsistent state when a transaction terminates abnormally. Recent work has attempted to fix known, easily reproducible error modes that caused these sorts of issues and to add rollback logic so that the inconsistencies are normally cleaned up immediately on request threads [1]. Even so, it is important to acknowledge that there is no existing mechanism that can ensure that the index and associated objects are eventually consistent when admitting the possibility of unidentified bugs or fault scenarios (e.g. the process dies in the middle of a transaction and there is no opportunity to rollback).
For clusters that have systematically reproduced the types of bugs/conditions that evoke the aforementioned inconsistencies (typically due to particular patterns of client behavior), there can be significant consequences. Most obviously, extraneous objects and index entries can cause bloat in space consumption. Secondarily, the nature of the bogus index entries that are left behind in these scenarios gives rise to severe performance degradation for bucket listings.
This intent of this issue is to propose new radosgw-admin commands for the purpose of identifying and cleaning up the aforementioned inconsistencies with versioned bucket indexes and associated objects.
Updated by Cory Snyder 10 months ago
- Related to Bug #61359: Consistency bugs with OLH objects added
Updated by Casey Bodley 8 months ago
- Status changed from New to Pending Backport
Updated by Backport Bot 8 months ago
- Copied to Backport #62943: quincy: New radosgw-admin commands to cleanup leftover OLH index entries and unlinked instance objects added
Updated by Backport Bot 8 months ago
- Copied to Backport #62944: reef: New radosgw-admin commands to cleanup leftover OLH index entries and unlinked instance objects added
Updated by Backport Bot 8 months ago
- Copied to Backport #62945: pacific: New radosgw-admin commands to cleanup leftover OLH index entries and unlinked instance objects added
Updated by Konstantin Shalygin 7 months ago
- Tracker changed from Feature to Bug
- Status changed from Pending Backport to Resolved
- Target version set to v19.0.0
- % Done changed from 0 to 100
- Regression set to No
- Severity set to 3 - minor
Updated by Cory Snyder 6 months ago
- Related to Bug #62760: versioned bucket stats can be incorrect after reshard or radosgw-admin bucket check --fix added