Project

General

Profile

Actions

Bug #36512

closed

Lifecycle doesn't remove delete markers

Added by Sean Purdy over 5 years ago. Updated about 3 years ago.

Status:
Resolved
Priority:
High
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
nautilus, mimic, luminous
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

I have a versioned bucket with a lifecycle. Files get deleted and delete markers added. A lifecycle configuration deletes the non-current versions, but doesn't remove the delete markers.

Lifecycle configuration:

{ 
  "Rules": [
    {
      "ID": "Test expiry 2",
      "Status": "Enabled",
      "Prefix": "",
      "NoncurrentVersionExpiration": {
        "NoncurrentDays": 1
      },
      "Expiration": {
        "ExpiredObjectDeleteMarker": true
      }
    }
  ]
}

I delete a file in the bucket. Delete marker is added:

{
    "DeleteMarkers": [
        {
            "Owner": {
                "DisplayName": "Sean Test", 
                "ID": "seantest" 
            }, 
            "IsLatest": true, 
            "VersionId": "H7pb8jPght20pKQd.b-m3nZPEvdW.Vd", 
            "Key": "00008000.txt", 
            "LastModified": "2018-10-16T15:29:48.105Z" 
        }
    ], 
    "Versions": [
        {
            "LastModified": "2018-10-16T14:36:54.717Z", 
            "VersionId": "bZEYkUaJLcgVFEZCWAOZwvpPrCHBhRt", 
            "ETag": "\"f262eaa559613b0eaaf53983ff880ec9\"", 
            "StorageClass": "STANDARD", 
            "Key": "00008000.txt", 
            "Owner": {
                "DisplayName": "Sean Test", 
                "ID": "seantest" 
            }, 
            "IsLatest": false, 
            "Size": 243
        }
    ]
}

When the lifecycle triggers, I get this:

{
    "DeleteMarkers": [
        {
            "Owner": {
                "DisplayName": "Sean Test", 
                "ID": "seantest" 
            }, 
            "IsLatest": true, 
            "VersionId": "H7pb8jPght20pKQd.b-m3nZPEvdW.Vd", 
            "Key": "00008000.txt", 
            "LastModified": "2018-10-16T15:29:48.105Z" 
        }
    ]
}

So why didn't the lifecycle remove the delete markers? The non-current version was removed.

Tested on 12.2.7 on debian stretch and 12.2.8 on debian jessie.

Thanks,

Sean Purdy


Related issues 3 (0 open3 closed)

Copied to rgw - Backport #38882: mimic: Lifecycle doesn't remove delete markersRejectedActions
Copied to rgw - Backport #38883: nautilus: Lifecycle doesn't remove delete markersResolvedAbhishek LekshmananActions
Copied to rgw - Backport #38884: luminous: Lifecycle doesn't remove delete markersResolvedActions
Actions #1

Updated by Casey Bodley over 5 years ago

  • Priority changed from Normal to High
Actions #2

Updated by Sean Purdy over 5 years ago

More specifically, it seems this behaviour happens only if you have autosharding enabled and a reshard has happened.

Actions #3

Updated by Abhishek Lekshmanan over 5 years ago

  • Assignee set to Abhishek Lekshmanan
Actions #4

Updated by Matt Benjamin about 5 years ago

  • Status changed from New to Triaged
Actions #5

Updated by Abhishek Lekshmanan about 5 years ago

  • Status changed from Triaged to Fix Under Review
Actions #6

Updated by Casey Bodley about 5 years ago

  • Status changed from Fix Under Review to 7
Actions #7

Updated by Sage Weil about 5 years ago

  • Backport set to nautilus
Actions #8

Updated by Abhishek Lekshmanan about 5 years ago

  • Backport changed from nautilus to nautilus, mimic, luminous
Actions #9

Updated by Casey Bodley about 5 years ago

  • Status changed from 7 to Pending Backport
Actions #10

Updated by Nathan Cutler about 5 years ago

  • Copied to Backport #38882: mimic: Lifecycle doesn't remove delete markers added
Actions #11

Updated by Nathan Cutler about 5 years ago

  • Copied to Backport #38883: nautilus: Lifecycle doesn't remove delete markers added
Actions #12

Updated by Nathan Cutler about 5 years ago

  • Copied to Backport #38884: luminous: Lifecycle doesn't remove delete markers added
Actions #13

Updated by Nathan Cutler about 5 years ago

  • Pull request ID set to 26564
Actions #14

Updated by Nathan Cutler almost 5 years ago

Backporting note from https://github.com/ceph/ceph/pull/29122

Since it's almost not possible to backport the entire PR #26564. The dependencies make it difficult and not proper to back port all of them.
I only backported the 2 commits that fixes the lifecycle not taking effect after resharding.
I think this is the main pain for ceph user, as you can't expect lifecycle to work with dynamic resharding enabled.

The other part of PR #26564, is about adding a command to radosgw-admin "radosgw-admin lc reshard fix" which can re-add the lifecycle to the old bucket.
This part is not proper for backport.
And user can always reset their lifecycle to the old bucket via commands like "s3cmd setlifecycle" as a workaround. I think we can leave this part for now.

Actions #15

Updated by Nathan Cutler about 3 years ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

Actions

Also available in: Atom PDF