Project

General

Profile

Bug #19587

rgw: put lifecycle configuration fails if Prefix is not set

Added by Ben Hines almost 2 years ago. Updated over 1 year ago.

Status:
Resolved
Priority:
High
Target version:
-
Start date:
04/12/2017
Due date:
% Done:

0%

Source:
Community (user)
Tags:
Backport:
luminous
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:

Description

1. Attempt to put a lifeycle configuration Using the following boto3 code: (full source attached)

bucket_lifecycle = s3.BucketLifecycle(bucket)
response = client.put_bucket_lifecycle_configuration(
    Bucket=bucket,
    LifecycleConfiguration={
        'Rules': [
            {
#'Prefix': '',
                'Expiration': {
                    'Days':1,
                },
                'ID': '0',
                'Status': 'Enabled',
            },
        ]

    }
)

2. Result:

Traceback (most recent call last):
  File "test_lifecycle.py", line 50, in <module>
    'Status': 'Enabled',
  File "/usr/lib/python2.7/site-packages/botocore/client.py", line 253, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/lib/python2.7/site-packages/botocore/client.py", line 543, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the PutBucketLifecycleConfiguration operation: Unknown

If the 'Prefix' is uncommented, the put is accepted.

Per Amazon S3 api docs: http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTlifecycle.html 'Prefix' is an optional field. I'm fine with just passing a blank prefix, but the AccessDenied error gave me no clue as to what the problem was. RGW should probably return some other error if the XML Parsing fails.

Debug log output:

2017-04-11 22:05:54.554247 7f3c641be700  5 Getting permissions identity=RGWThirdPartyAccountAuthApplier() -> RGWLocalAuthApplier(acct_user=zone.user, acct_name=Zone User, subuser=, perm_mask=15, is_admin=) owner=smbuildmachine perm=0
2017-04-11 22:05:54.554253 7f3c641be700 10  identity=RGWThirdPartyAccountAuthApplier() -> RGWLocalAuthApplier(acct_user=zone.user, acct_name=Zone User, subuser=, perm_mask=15, is_admin=) requested perm (type)=8, policy perm=0, user_perm_mask=8, acl perm=0
2017-04-11 22:05:54.554258 7f3c641be700  2 overriding permissions due to system operation
2017-04-11 22:05:54.554263 7f3c641be700  2 req 26:0.000636:s3:PUT /bentest:put_lifecycle:verifying op params
2017-04-11 22:05:54.554267 7f3c641be700  2 req 26:0.000642:s3:PUT /bentest:put_lifecycle:pre-executing
2017-04-11 22:05:54.554272 7f3c641be700  2 req 26:0.000647:s3:PUT /bentest:put_lifecycle:executing
2017-04-11 22:05:54.554293 7f3c641be700 15 read len=183 data=<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Rule><Status>Enabled</Status><Expiration><Days>1</Days></Expiration><ID>0</ID></Rule></LifecycleConfiguration>
2017-04-11 22:05:54.554365 7f3c641be700  2 req 26:0.000738:s3:PUT /bentest:put_lifecycle:completing
2017-04-11 22:05:54.554680 7f3c641be700  2 req 26:0.001053:s3:PUT /bentest:put_lifecycle:op status=-13
2017-04-11 22:05:54.554718 7f3c641be700  2 req 26:0.001092:s3:PUT /bentest:put_lifecycle:http status=403
2017-04-11 22:05:54.554731 7f3c641be700  1 ====== req done req=0x7f3c641b8340 op status=-13 http_status=403 ======
2017-04-11 22:05:54.554767 7f3c641be700 20 process_request() returned -13

Debug log output, with Prefix;

0
2017-04-11 22:06:37.362867 7f3c649bf700  2 overriding permissions due to system operation
2017-04-11 22:06:37.362870 7f3c649bf700  2 req 28:0.000794:s3:PUT /bentest:put_lifecycle:verifying op params
2017-04-11 22:06:37.362876 7f3c649bf700  2 req 28:0.000798:s3:PUT /bentest:put_lifecycle:pre-executing
2017-04-11 22:06:37.362881 7f3c649bf700  2 req 28:0.000805:s3:PUT /bentest:put_lifecycle:executing
2017-04-11 22:06:37.362922 7f3c649bf700 15 read len=193 data=<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Rule><Status>Enabled</Status><Prefix /><Expiration><Days>1</Days></Expiration><ID>0</ID></Rule></LifecycleConfiguration>
2017-04-11 22:06:37.363001 7f3c649bf700 15 Old LifecycleConfiguration<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Rule><ID>0</ID><Prefix></Prefix><Status>Enabled</Status><Expiration><Days>1</Days></Expiration></Rule></LifecycleConfiguration>
2017-04-11 22:06:37.363020 7f3c649bf700 15 New LifecycleConfiguration:<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Rule><ID>0</ID><Prefix></Prefix><Status>Enabled</Status><Expiration><Days>1</Days></Expiration></Rule></LifecycleConfiguration>
2017-04-11 22:06:37.390231 7f3c649bf700 10 cache put: name=.rgw+.bucket.meta.bentest:default.653959.6 info.flags=0x17
2017-04-11 22:06:37.390308 7f3c649bf700 10 moving .rgw+.bucket.meta.bentest:default.653959.6 to cache LRU end
2017-04-11 22:06:37.390330 7f3c649bf700 10 updating xattr: name=user.rgw.acl bl.length()=798
2017-04-11 22:06:37.390340 7f3c649bf700 10 updating xattr: name=user.rgw.idtag bl.length()=0
2017-04-11 22:06:37.390344 7f3c649bf700 10 updating xattr: name=user.rgw.lc bl.length()=52
2017-04-11 22:06:37.390390 7f3c649bf700 10 distributing notification oid=notify.6 bl.length()=1390
2017-04-11 22:06:37.391679 7f3d022fa700 10 RGWWatcher::handle_notify()  notify_id 582809882198022 cookie 139900567180544 notifier 70181692 bl.length()=1390
2017-04-11 22:06:37.391719 7f3d022fa700 10 cache put: name=.rgw+.bucket.meta.bentest:default.653959.6 info.flags=0x17
2017-04-11 22:06:37.391726 7f3d022fa700 10 moving .rgw+.bucket.meta.bentest:default.653959.6 to cache LRU end
2017-04-11 22:06:37.391734 7f3d022fa700 10 updating xattr: name=user.rgw.acl bl.length()=798
2017-04-11 22:06:37.391736 7f3d022fa700 10 updating xattr: name=user.rgw.idtag bl.length()=0
2017-04-11 22:06:37.391737 7f3d022fa700 10 updating xattr: name=user.rgw.lc bl.length()=52
2017-04-11 22:06:37.444068 7f3c649bf700  2 req 28:0.081988:s3:PUT /bentest:put_lifecycle:completing
2017-04-11 22:06:37.444406 7f3c649bf700  2 req 28:0.082328:s3:PUT /bentest:put_lifecycle:op status=0
2017-04-11 22:06:37.444432 7f3c649bf700  2 req 28:0.082356:s3:PUT /bentest:put_lifecycle:http status=200

rgw-lifecycle-repro.py View (941 Bytes) Ben Hines, 04/12/2017 05:07 AM


Related issues

Copied to rgw - Backport #21139: luminous: rgw: put lifecycle configuration fails if Prefix is not set Resolved

History

#1 Updated by Yehuda Sadeh almost 2 years ago

  • Subject changed from rgw putt lifecycle configuration fails if Prefix is not set to rgw: put lifecycle configuration fails if Prefix is not set

#2 Updated by Yehuda Sadeh almost 2 years ago

  • Description updated (diff)

#3 Updated by Ben Hines almost 2 years ago

Thanks for fixing formatting, i always mess that up. (and edit isn't allowed for me :)

#4 Updated by Josh Durgin almost 2 years ago

  • Project changed from Ceph to rgw
  • Category deleted (22)

#5 Updated by Yehuda Sadeh over 1 year ago

  • Priority changed from Normal to High

#6 Updated by Yehuda Sadeh over 1 year ago

  • Assignee set to Ali Maredia

#7 Updated by Matt Benjamin over 1 year ago

  • Status changed from New to In Progress

#8 Updated by Orit Wasserman over 1 year ago

  • Assignee changed from Ali Maredia to Abhishek Lekshmanan

#9 Updated by Abhishek Lekshmanan over 1 year ago

This particular example fails in AWS as well, either you need a prefix or a Filter, I have added support for Filter tag in https://github.com/ceph/ceph/pull/16818 and also supporting other aws like features like creating an ID if one is not provided. So either of Filter or Prefix argument has to be provided, an Empty Filter is ok, or an Empty Prefix will also succeed similar to S3's behaviour, but both fields being empty would throw a malformed XML

#10 Updated by Abhishek Lekshmanan over 1 year ago

  • Backport set to luminous

#11 Updated by Abhishek Lekshmanan over 1 year ago

  • Status changed from In Progress to Need Review

#12 Updated by Ben Hines over 1 year ago

Abhishek Lekshmanan wrote:

This particular example fails in AWS as well, either you need a prefix or a Filter, I have added support for Filter tag in https://github.com/ceph/ceph/pull/16818 and also supporting other aws like features like creating an ID if one is not provided. So either of Filter or Prefix argument has to be provided, an Empty Filter is ok, or an Empty Prefix will also succeed similar to S3's behaviour, but both fields being empty would throw a malformed XML

I believe the actual failure (At least with kraken currently), is an 'AccessDenied' error. I don't have S3 to test out currently, but if there was a way to throw some other more correct error to the client (like malformed XML) it would be nice. Though, matching S3's behavior is also a worthy goal.

#14 Updated by Yuri Weinstein over 1 year ago

Abhishek Lekshmanan wrote:

master PR: https://github.com/ceph/ceph/pull/16818

merged

#15 Updated by Yehuda Sadeh over 1 year ago

  • Status changed from Need Review to Pending Backport

#16 Updated by Nathan Cutler over 1 year ago

  • Copied to Backport #21139: luminous: rgw: put lifecycle configuration fails if Prefix is not set added

#17 Updated by Abhishek Lekshmanan over 1 year ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF