Project

General

Profile

Actions

Bug #19587

closed

rgw: put lifecycle configuration fails if Prefix is not set

Added by Ben Hines about 7 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
High
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
Backport:
luminous
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

1. Attempt to put a lifeycle configuration Using the following boto3 code: (full source attached)

bucket_lifecycle = s3.BucketLifecycle(bucket)
response = client.put_bucket_lifecycle_configuration(
    Bucket=bucket,
    LifecycleConfiguration={
        'Rules': [
            {
#'Prefix': '',
                'Expiration': {
                    'Days':1,
                },
                'ID': '0',
                'Status': 'Enabled',
            },
        ]

    }
)

2. Result:

Traceback (most recent call last):
  File "test_lifecycle.py", line 50, in <module>
    'Status': 'Enabled',
  File "/usr/lib/python2.7/site-packages/botocore/client.py", line 253, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/lib/python2.7/site-packages/botocore/client.py", line 543, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the PutBucketLifecycleConfiguration operation: Unknown

If the 'Prefix' is uncommented, the put is accepted.

Per Amazon S3 api docs: http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTlifecycle.html 'Prefix' is an optional field. I'm fine with just passing a blank prefix, but the AccessDenied error gave me no clue as to what the problem was. RGW should probably return some other error if the XML Parsing fails.

Debug log output:

2017-04-11 22:05:54.554247 7f3c641be700  5 Getting permissions identity=RGWThirdPartyAccountAuthApplier() -> RGWLocalAuthApplier(acct_user=zone.user, acct_name=Zone User, subuser=, perm_mask=15, is_admin=) owner=smbuildmachine perm=0
2017-04-11 22:05:54.554253 7f3c641be700 10  identity=RGWThirdPartyAccountAuthApplier() -> RGWLocalAuthApplier(acct_user=zone.user, acct_name=Zone User, subuser=, perm_mask=15, is_admin=) requested perm (type)=8, policy perm=0, user_perm_mask=8, acl perm=0
2017-04-11 22:05:54.554258 7f3c641be700  2 overriding permissions due to system operation
2017-04-11 22:05:54.554263 7f3c641be700  2 req 26:0.000636:s3:PUT /bentest:put_lifecycle:verifying op params
2017-04-11 22:05:54.554267 7f3c641be700  2 req 26:0.000642:s3:PUT /bentest:put_lifecycle:pre-executing
2017-04-11 22:05:54.554272 7f3c641be700  2 req 26:0.000647:s3:PUT /bentest:put_lifecycle:executing
2017-04-11 22:05:54.554293 7f3c641be700 15 read len=183 data=<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Rule><Status>Enabled</Status><Expiration><Days>1</Days></Expiration><ID>0</ID></Rule></LifecycleConfiguration>
2017-04-11 22:05:54.554365 7f3c641be700  2 req 26:0.000738:s3:PUT /bentest:put_lifecycle:completing
2017-04-11 22:05:54.554680 7f3c641be700  2 req 26:0.001053:s3:PUT /bentest:put_lifecycle:op status=-13
2017-04-11 22:05:54.554718 7f3c641be700  2 req 26:0.001092:s3:PUT /bentest:put_lifecycle:http status=403
2017-04-11 22:05:54.554731 7f3c641be700  1 ====== req done req=0x7f3c641b8340 op status=-13 http_status=403 ======
2017-04-11 22:05:54.554767 7f3c641be700 20 process_request() returned -13

Debug log output, with Prefix;

0
2017-04-11 22:06:37.362867 7f3c649bf700  2 overriding permissions due to system operation
2017-04-11 22:06:37.362870 7f3c649bf700  2 req 28:0.000794:s3:PUT /bentest:put_lifecycle:verifying op params
2017-04-11 22:06:37.362876 7f3c649bf700  2 req 28:0.000798:s3:PUT /bentest:put_lifecycle:pre-executing
2017-04-11 22:06:37.362881 7f3c649bf700  2 req 28:0.000805:s3:PUT /bentest:put_lifecycle:executing
2017-04-11 22:06:37.362922 7f3c649bf700 15 read len=193 data=<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Rule><Status>Enabled</Status><Prefix /><Expiration><Days>1</Days></Expiration><ID>0</ID></Rule></LifecycleConfiguration>
2017-04-11 22:06:37.363001 7f3c649bf700 15 Old LifecycleConfiguration<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Rule><ID>0</ID><Prefix></Prefix><Status>Enabled</Status><Expiration><Days>1</Days></Expiration></Rule></LifecycleConfiguration>
2017-04-11 22:06:37.363020 7f3c649bf700 15 New LifecycleConfiguration:<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Rule><ID>0</ID><Prefix></Prefix><Status>Enabled</Status><Expiration><Days>1</Days></Expiration></Rule></LifecycleConfiguration>
2017-04-11 22:06:37.390231 7f3c649bf700 10 cache put: name=.rgw+.bucket.meta.bentest:default.653959.6 info.flags=0x17
2017-04-11 22:06:37.390308 7f3c649bf700 10 moving .rgw+.bucket.meta.bentest:default.653959.6 to cache LRU end
2017-04-11 22:06:37.390330 7f3c649bf700 10 updating xattr: name=user.rgw.acl bl.length()=798
2017-04-11 22:06:37.390340 7f3c649bf700 10 updating xattr: name=user.rgw.idtag bl.length()=0
2017-04-11 22:06:37.390344 7f3c649bf700 10 updating xattr: name=user.rgw.lc bl.length()=52
2017-04-11 22:06:37.390390 7f3c649bf700 10 distributing notification oid=notify.6 bl.length()=1390
2017-04-11 22:06:37.391679 7f3d022fa700 10 RGWWatcher::handle_notify()  notify_id 582809882198022 cookie 139900567180544 notifier 70181692 bl.length()=1390
2017-04-11 22:06:37.391719 7f3d022fa700 10 cache put: name=.rgw+.bucket.meta.bentest:default.653959.6 info.flags=0x17
2017-04-11 22:06:37.391726 7f3d022fa700 10 moving .rgw+.bucket.meta.bentest:default.653959.6 to cache LRU end
2017-04-11 22:06:37.391734 7f3d022fa700 10 updating xattr: name=user.rgw.acl bl.length()=798
2017-04-11 22:06:37.391736 7f3d022fa700 10 updating xattr: name=user.rgw.idtag bl.length()=0
2017-04-11 22:06:37.391737 7f3d022fa700 10 updating xattr: name=user.rgw.lc bl.length()=52
2017-04-11 22:06:37.444068 7f3c649bf700  2 req 28:0.081988:s3:PUT /bentest:put_lifecycle:completing
2017-04-11 22:06:37.444406 7f3c649bf700  2 req 28:0.082328:s3:PUT /bentest:put_lifecycle:op status=0
2017-04-11 22:06:37.444432 7f3c649bf700  2 req 28:0.082356:s3:PUT /bentest:put_lifecycle:http status=200

Files

rgw-lifecycle-repro.py (941 Bytes) rgw-lifecycle-repro.py Ben Hines, 04/12/2017 05:07 AM

Related issues 1 (0 open1 closed)

Copied to rgw - Backport #21139: luminous: rgw: put lifecycle configuration fails if Prefix is not setResolvedNathan CutlerActions
Actions #1

Updated by Yehuda Sadeh about 7 years ago

  • Subject changed from rgw putt lifecycle configuration fails if Prefix is not set to rgw: put lifecycle configuration fails if Prefix is not set
Actions #2

Updated by Yehuda Sadeh about 7 years ago

  • Description updated (diff)
Actions #3

Updated by Ben Hines about 7 years ago

Thanks for fixing formatting, i always mess that up. (and edit isn't allowed for me :)

Actions #4

Updated by Josh Durgin about 7 years ago

  • Project changed from Ceph to rgw
  • Category deleted (22)
Actions #5

Updated by Yehuda Sadeh about 7 years ago

  • Priority changed from Normal to High
Actions #6

Updated by Yehuda Sadeh almost 7 years ago

  • Assignee set to Ali Maredia
Actions #7

Updated by Matt Benjamin almost 7 years ago

  • Status changed from New to In Progress
Actions #8

Updated by Orit Wasserman over 6 years ago

  • Assignee changed from Ali Maredia to Abhishek Lekshmanan
Actions #9

Updated by Abhishek Lekshmanan over 6 years ago

This particular example fails in AWS as well, either you need a prefix or a Filter, I have added support for Filter tag in https://github.com/ceph/ceph/pull/16818 and also supporting other aws like features like creating an ID if one is not provided. So either of Filter or Prefix argument has to be provided, an Empty Filter is ok, or an Empty Prefix will also succeed similar to S3's behaviour, but both fields being empty would throw a malformed XML

Actions #10

Updated by Abhishek Lekshmanan over 6 years ago

  • Backport set to luminous
Actions #11

Updated by Abhishek Lekshmanan over 6 years ago

  • Status changed from In Progress to Fix Under Review
Actions #12

Updated by Ben Hines over 6 years ago

Abhishek Lekshmanan wrote:

This particular example fails in AWS as well, either you need a prefix or a Filter, I have added support for Filter tag in https://github.com/ceph/ceph/pull/16818 and also supporting other aws like features like creating an ID if one is not provided. So either of Filter or Prefix argument has to be provided, an Empty Filter is ok, or an Empty Prefix will also succeed similar to S3's behaviour, but both fields being empty would throw a malformed XML

I believe the actual failure (At least with kraken currently), is an 'AccessDenied' error. I don't have S3 to test out currently, but if there was a way to throw some other more correct error to the client (like malformed XML) it would be nice. Though, matching S3's behavior is also a worthy goal.

Actions #14

Updated by Yuri Weinstein over 6 years ago

Abhishek Lekshmanan wrote:

master PR: https://github.com/ceph/ceph/pull/16818

merged

Actions #15

Updated by Yehuda Sadeh over 6 years ago

  • Status changed from Fix Under Review to Pending Backport
Actions #16

Updated by Nathan Cutler over 6 years ago

  • Copied to Backport #21139: luminous: rgw: put lifecycle configuration fails if Prefix is not set added
Actions #17

Updated by Abhishek Lekshmanan over 6 years ago

  • Status changed from Pending Backport to Resolved
Actions

Also available in: Atom PDF