Bug #19587
closedrgw: put lifecycle configuration fails if Prefix is not set
0%
Description
1. Attempt to put a lifeycle configuration Using the following boto3 code: (full source attached)
bucket_lifecycle = s3.BucketLifecycle(bucket) response = client.put_bucket_lifecycle_configuration( Bucket=bucket, LifecycleConfiguration={ 'Rules': [ { #'Prefix': '', 'Expiration': { 'Days':1, }, 'ID': '0', 'Status': 'Enabled', }, ] } )
2. Result:
Traceback (most recent call last): File "test_lifecycle.py", line 50, in <module> 'Status': 'Enabled', File "/usr/lib/python2.7/site-packages/botocore/client.py", line 253, in _api_call return self._make_api_call(operation_name, kwargs) File "/usr/lib/python2.7/site-packages/botocore/client.py", line 543, in _make_api_call raise error_class(parsed_response, operation_name) botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the PutBucketLifecycleConfiguration operation: Unknown
If the 'Prefix' is uncommented, the put is accepted.
Per Amazon S3 api docs: http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTlifecycle.html 'Prefix' is an optional field. I'm fine with just passing a blank prefix, but the AccessDenied error gave me no clue as to what the problem was. RGW should probably return some other error if the XML Parsing fails.
Debug log output:
2017-04-11 22:05:54.554247 7f3c641be700 5 Getting permissions identity=RGWThirdPartyAccountAuthApplier() -> RGWLocalAuthApplier(acct_user=zone.user, acct_name=Zone User, subuser=, perm_mask=15, is_admin=) owner=smbuildmachine perm=0 2017-04-11 22:05:54.554253 7f3c641be700 10 identity=RGWThirdPartyAccountAuthApplier() -> RGWLocalAuthApplier(acct_user=zone.user, acct_name=Zone User, subuser=, perm_mask=15, is_admin=) requested perm (type)=8, policy perm=0, user_perm_mask=8, acl perm=0 2017-04-11 22:05:54.554258 7f3c641be700 2 overriding permissions due to system operation 2017-04-11 22:05:54.554263 7f3c641be700 2 req 26:0.000636:s3:PUT /bentest:put_lifecycle:verifying op params 2017-04-11 22:05:54.554267 7f3c641be700 2 req 26:0.000642:s3:PUT /bentest:put_lifecycle:pre-executing 2017-04-11 22:05:54.554272 7f3c641be700 2 req 26:0.000647:s3:PUT /bentest:put_lifecycle:executing 2017-04-11 22:05:54.554293 7f3c641be700 15 read len=183 data=<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Rule><Status>Enabled</Status><Expiration><Days>1</Days></Expiration><ID>0</ID></Rule></LifecycleConfiguration> 2017-04-11 22:05:54.554365 7f3c641be700 2 req 26:0.000738:s3:PUT /bentest:put_lifecycle:completing 2017-04-11 22:05:54.554680 7f3c641be700 2 req 26:0.001053:s3:PUT /bentest:put_lifecycle:op status=-13 2017-04-11 22:05:54.554718 7f3c641be700 2 req 26:0.001092:s3:PUT /bentest:put_lifecycle:http status=403 2017-04-11 22:05:54.554731 7f3c641be700 1 ====== req done req=0x7f3c641b8340 op status=-13 http_status=403 ====== 2017-04-11 22:05:54.554767 7f3c641be700 20 process_request() returned -13
Debug log output, with Prefix;
0 2017-04-11 22:06:37.362867 7f3c649bf700 2 overriding permissions due to system operation 2017-04-11 22:06:37.362870 7f3c649bf700 2 req 28:0.000794:s3:PUT /bentest:put_lifecycle:verifying op params 2017-04-11 22:06:37.362876 7f3c649bf700 2 req 28:0.000798:s3:PUT /bentest:put_lifecycle:pre-executing 2017-04-11 22:06:37.362881 7f3c649bf700 2 req 28:0.000805:s3:PUT /bentest:put_lifecycle:executing 2017-04-11 22:06:37.362922 7f3c649bf700 15 read len=193 data=<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Rule><Status>Enabled</Status><Prefix /><Expiration><Days>1</Days></Expiration><ID>0</ID></Rule></LifecycleConfiguration> 2017-04-11 22:06:37.363001 7f3c649bf700 15 Old LifecycleConfiguration<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Rule><ID>0</ID><Prefix></Prefix><Status>Enabled</Status><Expiration><Days>1</Days></Expiration></Rule></LifecycleConfiguration> 2017-04-11 22:06:37.363020 7f3c649bf700 15 New LifecycleConfiguration:<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Rule><ID>0</ID><Prefix></Prefix><Status>Enabled</Status><Expiration><Days>1</Days></Expiration></Rule></LifecycleConfiguration> 2017-04-11 22:06:37.390231 7f3c649bf700 10 cache put: name=.rgw+.bucket.meta.bentest:default.653959.6 info.flags=0x17 2017-04-11 22:06:37.390308 7f3c649bf700 10 moving .rgw+.bucket.meta.bentest:default.653959.6 to cache LRU end 2017-04-11 22:06:37.390330 7f3c649bf700 10 updating xattr: name=user.rgw.acl bl.length()=798 2017-04-11 22:06:37.390340 7f3c649bf700 10 updating xattr: name=user.rgw.idtag bl.length()=0 2017-04-11 22:06:37.390344 7f3c649bf700 10 updating xattr: name=user.rgw.lc bl.length()=52 2017-04-11 22:06:37.390390 7f3c649bf700 10 distributing notification oid=notify.6 bl.length()=1390 2017-04-11 22:06:37.391679 7f3d022fa700 10 RGWWatcher::handle_notify() notify_id 582809882198022 cookie 139900567180544 notifier 70181692 bl.length()=1390 2017-04-11 22:06:37.391719 7f3d022fa700 10 cache put: name=.rgw+.bucket.meta.bentest:default.653959.6 info.flags=0x17 2017-04-11 22:06:37.391726 7f3d022fa700 10 moving .rgw+.bucket.meta.bentest:default.653959.6 to cache LRU end 2017-04-11 22:06:37.391734 7f3d022fa700 10 updating xattr: name=user.rgw.acl bl.length()=798 2017-04-11 22:06:37.391736 7f3d022fa700 10 updating xattr: name=user.rgw.idtag bl.length()=0 2017-04-11 22:06:37.391737 7f3d022fa700 10 updating xattr: name=user.rgw.lc bl.length()=52 2017-04-11 22:06:37.444068 7f3c649bf700 2 req 28:0.081988:s3:PUT /bentest:put_lifecycle:completing 2017-04-11 22:06:37.444406 7f3c649bf700 2 req 28:0.082328:s3:PUT /bentest:put_lifecycle:op status=0 2017-04-11 22:06:37.444432 7f3c649bf700 2 req 28:0.082356:s3:PUT /bentest:put_lifecycle:http status=200
Files
Updated by Yehuda Sadeh about 7 years ago
- Subject changed from rgw putt lifecycle configuration fails if Prefix is not set to rgw: put lifecycle configuration fails if Prefix is not set
Updated by Ben Hines about 7 years ago
Thanks for fixing formatting, i always mess that up. (and edit isn't allowed for me :)
Updated by Josh Durgin about 7 years ago
- Project changed from Ceph to rgw
- Category deleted (
22)
Updated by Matt Benjamin almost 7 years ago
- Status changed from New to In Progress
Updated by Orit Wasserman almost 7 years ago
- Assignee changed from Ali Maredia to Abhishek Lekshmanan
Updated by Abhishek Lekshmanan over 6 years ago
This particular example fails in AWS as well, either you need a prefix or a Filter, I have added support for Filter tag in https://github.com/ceph/ceph/pull/16818 and also supporting other aws like features like creating an ID if one is not provided. So either of Filter or Prefix argument has to be provided, an Empty Filter is ok, or an Empty Prefix will also succeed similar to S3's behaviour, but both fields being empty would throw a malformed XML
Updated by Abhishek Lekshmanan over 6 years ago
- Status changed from In Progress to Fix Under Review
Updated by Ben Hines over 6 years ago
Abhishek Lekshmanan wrote:
This particular example fails in AWS as well, either you need a prefix or a Filter, I have added support for Filter tag in https://github.com/ceph/ceph/pull/16818 and also supporting other aws like features like creating an ID if one is not provided. So either of Filter or Prefix argument has to be provided, an Empty Filter is ok, or an Empty Prefix will also succeed similar to S3's behaviour, but both fields being empty would throw a malformed XML
I believe the actual failure (At least with kraken currently), is an 'AccessDenied' error. I don't have S3 to test out currently, but if there was a way to throw some other more correct error to the client (like malformed XML) it would be nice. Though, matching S3's behavior is also a worthy goal.
Updated by Abhishek Lekshmanan over 6 years ago
master PR: https://github.com/ceph/ceph/pull/16818
Updated by Yuri Weinstein over 6 years ago
Updated by Yehuda Sadeh over 6 years ago
- Status changed from Fix Under Review to Pending Backport
Updated by Nathan Cutler over 6 years ago
- Copied to Backport #21139: luminous: rgw: put lifecycle configuration fails if Prefix is not set added
Updated by Abhishek Lekshmanan over 6 years ago
- Status changed from Pending Backport to Resolved