Bug #47586
Able to circumvent S3 Object Lock using deleteobjects command
0%
Description
Ceph supports the S3 object lock which prevents objects being deleted until their retention date has expired, however it appears to be possible to circumvent this and delete objects before their retention date has passed by using the option to delete multiple objects at once.
See the following example using the AWS CLI to confirm this.
Create an object
$ aws --profile=ceph --endpoint-url https://ceph s3api put-object --bucket andrew-test --key ceph-bug --object-lock-mode COMPLIANCE --object-lock-retain-until "2020-09-22 16:00:00"
{
"ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"",
"VersionId": "-cBhsp3jHdeWAgxH9Grd.ZCRHsyT8n-"
}
Confirm object lock
$ aws --profile=ceph --endpoint-url https://ceph s3api get-object-retention --bucket andrew-test --key ceph-bug
{
"Retention": {
"Mode": "COMPLIANCE",
"RetainUntilDate": "2020-09-22T16:00:00.000000000Z"
}
}
Attempt to delete the version using a single delete
$ aws --profile=ceph --endpoint-url https://ceph s3api delete-object --bucket andrew-test --key ceph-bug --version-id=-cBhsp3jHdeWAgxH9Grd.ZCRHsyT8n-
An error occurred (AccessDenied) when calling the DeleteObject operation: Unknown
This correctly fails.
Attempt to delete the version using a multiple delete
$ aws --profile=ceph --endpoint-url https://ceph s3api delete-objects --bucket andrew-test --delete "Objects=[{Key=ceph-bug,VersionId=-cBhsp3jHdeWAgxH9Grd.ZCRHsyT8n-}],Quiet=false"
{
"Deleted": [
{
"Key": "ceph-bug",
"VersionId": "-cBhsp3jHdeWAgxH9Grd.ZCRHsyT8n-"
}
]
}
This succeeds and deletes the object version, despite it still being before the retention date.
Related issues
History
#1 Updated by Matt Benjamin over 3 years ago
- Project changed from Ceph to rgw
#2 Updated by Aleksandr Rudenko over 3 years ago
Hi,
I can confirm this behavior on 14.2.11.
#3 Updated by Mark Houghton over 3 years ago
Pull request to fix this: https://github.com/ceph/ceph/pull/37933
Updates to S3 tests: https://github.com/ceph/s3-tests/pull/365
#4 Updated by Nathan Cutler over 3 years ago
- Status changed from New to Fix Under Review
- Backport set to octopus, nautilus
- Pull request ID set to 37933
#5 Updated by J. Eric Ivancich over 3 years ago
- Status changed from Fix Under Review to Pending Backport
#6 Updated by Nathan Cutler over 3 years ago
- Copied to Backport #48423: nautilus: Able to circumvent S3 Object Lock using deleteobjects command added
#7 Updated by Nathan Cutler over 3 years ago
- Copied to Backport #48424: octopus: Able to circumvent S3 Object Lock using deleteobjects command added
#8 Updated by Simon Leinen almost 3 years ago
Isn't this bug more severe than "3 - minor"? It represents a clear violation of the promises of Object Lock. Backporting this to the Ceph versions actively used in the field (Nautilus and Octopus) would increase trust in Ceph again. For example, one of the ugly warnings in Veeam's object storage compatibility list1 could be eliminated:
Ceph (14.2.6 or later) [COMMUNITY ENTRY] POSSIBLE ISSUES (Date Format and >>> Immutability <<<)
(Sorry, I'm not personally involved in running the Ceph systems here, so I won't be able to backport or test the fix myself.)
#9 Updated by Robert Groenewald almost 3 years ago
This issue needs to picked up! This makes the reason why we are running Ceph almost unusable!
#10 Updated by Matt Benjamin almost 3 years ago
- Assignee set to Matt Benjamin
#11 Updated by Matt Benjamin almost 3 years ago
- Severity changed from 3 - minor to 2 - major
(for bug scrub 4/29: I will take ownership of this issue)
#12 Updated by Simon Leinen almost 3 years ago
Matt Benjamin wrote:
(for bug scrub 4/29: I will take ownership of this issue)
Thank you, Matt!
#13 Updated by Loïc Dachary almost 3 years ago
- Status changed from Pending Backport to Resolved
While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".
#14 Updated by J. Eric Ivancich 11 months ago
- Related to Bug #59474: Cannot delete object using multi-delete operation on a bucket with policy added