Bug #36512
closedLifecycle doesn't remove delete markers
0%
Description
I have a versioned bucket with a lifecycle. Files get deleted and delete markers added. A lifecycle configuration deletes the non-current versions, but doesn't remove the delete markers.
Lifecycle configuration:
{ "Rules": [ { "ID": "Test expiry 2", "Status": "Enabled", "Prefix": "", "NoncurrentVersionExpiration": { "NoncurrentDays": 1 }, "Expiration": { "ExpiredObjectDeleteMarker": true } } ] }
I delete a file in the bucket. Delete marker is added:
{ "DeleteMarkers": [ { "Owner": { "DisplayName": "Sean Test", "ID": "seantest" }, "IsLatest": true, "VersionId": "H7pb8jPght20pKQd.b-m3nZPEvdW.Vd", "Key": "00008000.txt", "LastModified": "2018-10-16T15:29:48.105Z" } ], "Versions": [ { "LastModified": "2018-10-16T14:36:54.717Z", "VersionId": "bZEYkUaJLcgVFEZCWAOZwvpPrCHBhRt", "ETag": "\"f262eaa559613b0eaaf53983ff880ec9\"", "StorageClass": "STANDARD", "Key": "00008000.txt", "Owner": { "DisplayName": "Sean Test", "ID": "seantest" }, "IsLatest": false, "Size": 243 } ] }
When the lifecycle triggers, I get this:
{ "DeleteMarkers": [ { "Owner": { "DisplayName": "Sean Test", "ID": "seantest" }, "IsLatest": true, "VersionId": "H7pb8jPght20pKQd.b-m3nZPEvdW.Vd", "Key": "00008000.txt", "LastModified": "2018-10-16T15:29:48.105Z" } ] }
So why didn't the lifecycle remove the delete markers? The non-current version was removed.
Tested on 12.2.7 on debian stretch and 12.2.8 on debian jessie.
Thanks,
Sean Purdy
Updated by Sean Purdy over 5 years ago
More specifically, it seems this behaviour happens only if you have autosharding enabled and a reshard has happened.
Updated by Abhishek Lekshmanan over 5 years ago
- Assignee set to Abhishek Lekshmanan
Updated by Abhishek Lekshmanan about 5 years ago
- Status changed from Triaged to Fix Under Review
Updated by Casey Bodley about 5 years ago
- Status changed from Fix Under Review to 7
Updated by Abhishek Lekshmanan about 5 years ago
- Backport changed from nautilus to nautilus, mimic, luminous
Updated by Casey Bodley about 5 years ago
- Status changed from 7 to Pending Backport
Updated by Nathan Cutler about 5 years ago
- Copied to Backport #38882: mimic: Lifecycle doesn't remove delete markers added
Updated by Nathan Cutler about 5 years ago
- Copied to Backport #38883: nautilus: Lifecycle doesn't remove delete markers added
Updated by Nathan Cutler about 5 years ago
- Copied to Backport #38884: luminous: Lifecycle doesn't remove delete markers added
Updated by Nathan Cutler almost 5 years ago
Backporting note from https://github.com/ceph/ceph/pull/29122
Since it's almost not possible to backport the entire PR #26564. The dependencies make it difficult and not proper to back port all of them.
I only backported the 2 commits that fixes the lifecycle not taking effect after resharding.
I think this is the main pain for ceph user, as you can't expect lifecycle to work with dynamic resharding enabled.
The other part of PR #26564, is about adding a command to radosgw-admin "radosgw-admin lc reshard fix" which can re-add the lifecycle to the old bucket.
This part is not proper for backport.
And user can always reset their lifecycle to the old bucket via commands like "s3cmd setlifecycle" as a workaround. I think we can leave this part for now.
Updated by Nathan Cutler about 3 years ago
- Status changed from Pending Backport to Resolved
While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".