Bug #39622: rgw:RGWGetBucketVersioning on non-existing bucket does not return NoSuchBucket (http=404) - rgw - Ceph

Actions

Copy link

Bug #39622

closed

rgw:RGWGetBucketVersioning on non-existing bucket does not return NoSuchBucket (http=404)

Added by Prashant D almost 5 years ago. Updated almost 5 years ago.

Status:

Duplicate

Priority:

High

Assignee:

Target version:

% Done:

Source:

Tags:

Backport:

mimic, luminous, nautilus

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

rgw

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

The issue is reproducible with downstream luminous and should be reproducible with master branch as well.

Steps to reproduce:
1. Get available buckets
$ python s3.list_bucket.py
Bucket List: ['bbq', 'my-av-bucket', 's3bucket']

-------------------------
s3.list_bucket.py
-------------------------
#!/usr/bin/python

import boto3
access_key = 's3user'
secret_key = 's3user'
s3 = boto3.client('s3', endpoint_url='http://rgw.default.abc.xyz.com:8080')
response = s3.list_buckets()
buckets = [bucket['Name'] for bucket in response['Buckets']]
print("Bucket List: %s" % buckets)
-------------------------

2. try to get bucket version for non-existing bucket
$ python s3.bucket_version.py {'ResponseMetadata': {'HTTPStatusCode': 200, 'RetryAttempts': 0, 'HostId': '', 'RequestId': 'tx000000000000000000063-005cd23c32-fac4-us-east', 'HTTPHeaders': {'date': 'Wed, 08 May 2019 02:17:22 GMT', 'content-length': '137', 'x-amz-request-id': 'tx000000000000000000063-005cd23c32-fac4-us-east', 'content-type': 'application/xml'}}}

---------------------------
s3.bucket_version.py
---------------------------
#!/usr/bin/python

import boto3
access_key = 's3user'
secret_key = 's3user'

Create an S3 client
s3 = boto3.client('s3', endpoint_url='http://rgw.default.abc.xyz.com:8080')

print(s3.get_bucket_versioning(Bucket='BUCKET'))
---------------------------

Actual results: {'ResponseMetadata': {'HTTPStatusCode': 200, 'RetryAttempts': 0, 'HostId': '', 'RequestId': 'tx000000000000000000063-005cd23c32-fac4-us-east', 'HTTPHeaders': {'date': 'Wed, 08 May 2019 02:17:22 GMT', 'content-length': '137', 'x-amz-request-id': 'tx000000000000000000063-005cd23c32-fac4-us-east', 'content-type': 'application/xml'}}}

Expected results:
botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the GetBucketVersioning operation: Unknown

Related issues 1 (0 open — 1 closed)

Actions

Copy link

Updated by Prashant D almost 5 years ago

Priority changed from Normal to High
Backport set to mimic, luminous, nautilus

Actions

Copy link

Updated by Prashant D almost 5 years ago

rgw debug logs :

2019-05-03 02:32:44.642317 7fea2734b700 1 ====== starting new request req=0x7fea27344f90 =====
2019-05-03 02:32:44.642415 7fea2734b700 2 req 90:0.000099::GET /BUCKET::initializing for trans_id = tx00000000000000000005a-005ccbe08c-fac4-us-east
2019-05-03 02:32:44.642491 7fea2734b700 10 rgw api priority: s3=6 s3website=5
2019-05-03 02:32:44.642502 7fea2734b700 10 host=rgw.default.abc.xyz.com
2019-05-03 02:32:44.642533 7fea2734b700 20 subdomain= domain=rgw.default.abc.xyz.com in_hosted_domain=1 in_hosted_domain_s3website=0
2019-05-03 02:32:44.642536 7fea2734b700 20 final domain/bucket subdomain= domain=rgw.default.abc.xyz.com in_hosted_domain=1 in_hosted_domain_s3website=0 s->info.domain=rgw.default.abc.xyz.com s->info.request_uri=/BUCKET
2019-05-03 02:32:44.642549 7fea2734b700 10 meta>> HTTP_X_AMZ_CONTENT_SHA256
2019-05-03 02:32:44.642562 7fea2734b700 10 meta>> HTTP_X_AMZ_DATE
2019-05-03 02:32:44.642566 7fea2734b700 10 x>> x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2019-05-03 02:32:44.642572 7fea2734b700 10 x>> x-amz-date:20190503T063244Z
2019-05-03 02:32:44.642660 7fea2734b700 20 get_handler handler=25RGWHandler_REST_Bucket_S3
2019-05-03 02:32:44.642705 7fea2734b700 10 handler=25RGWHandler_REST_Bucket_S3
2019-05-03 02:32:44.642707 7fea2734b700 2 req 90:0.000392:s3:GET /BUCKET::getting op 0
2019-05-03 02:32:44.642740 7fea2734b700 10 op=34RGWGetBucketVersioning_ObjStore_S3
2019-05-03 02:32:44.642750 7fea2734b700 2 req 90:0.000435:s3:GET /BUCKET:get_bucket_versioning:verifying requester
2019-05-03 02:32:44.642789 7fea2734b700 20 rgw::auth::StrategyRegistry::s3_main_strategy_t: trying rgw::auth::s3::AWSAuthStrategy
2019-05-03 02:32:44.642812 7fea2734b700 20 rgw::auth::s3::AWSAuthStrategy: trying rgw::auth::s3::S3AnonymousEngine
2019-05-03 02:32:44.642835 7fea2734b700 20 rgw::auth::s3::S3AnonymousEngine denied with reason=-1
2019-05-03 02:32:44.642838 7fea2734b700 20 rgw::auth::s3::AWSAuthStrategy: trying rgw::auth::s3::LocalEngine
2019-05-03 02:32:44.642910 7fea2734b700 10 v4 signature format = 8d80dab573fb61c2eb49a9b36f817199bd10d4504804a230a7273b963699a215
2019-05-03 02:32:44.642955 7fea2734b700 10 v4 credential format = s3user/20190503/us-east-1/s3/aws4_request
2019-05-03 02:32:44.642958 7fea2734b700 10 access key id = s3user
2019-05-03 02:32:44.642982 7fea2734b700 10 credential scope = 20190503/us-east-1/s3/aws4_request
2019-05-03 02:32:44.643060 7fea2734b700 10 canonical headers format = host:rgw.default.abc.xyz.com:8080
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20190503T063244Z

2019-05-03 02:32:44.643076 7fea2734b700 10 payload request hash = e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2019-05-03 02:32:44.643236 7fea2734b700 10 canonical request = GET
/BUCKET
versioning=
host:rgw.default.abc.xyz.com:8080
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20190503T063244Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2019-05-03 02:32:44.643239 7fea2734b700 10 canonical request hash = 17c64c14c926ae9de43f0b0647eb5314e7832d71c1b71af21917ba8e8e524dad
2019-05-03 02:32:44.643266 7fea2734b700 10 string to sign = AWS4-HMAC-SHA256
20190503T063244Z
20190503/us-east-1/s3/aws4_request
17c64c14c926ae9de43f0b0647eb5314e7832d71c1b71af21917ba8e8e524dad
2019-05-03 02:32:44.643530 7fea2734b700 10 date_k = 334903bc9c5a5391ad8ab9622a4c4e92abf85542eb3ed9679bb567573855d53c
2019-05-03 02:32:44.643544 7fea2734b700 10 region_k = 4702bb6eb942c6eb5a3d6864398c89eede904dc10e29435a75736b1a83c8d4ef
2019-05-03 02:32:44.643552 7fea2734b700 10 service_k = ceb7b7b0814dae28825134ee93a3832ecf648bd2f03699050bb2926c323ddd28
2019-05-03 02:32:44.643559 7fea2734b700 10 signing_k = b0f25cc31afeb5f83131c450e80bdb33f58d837ae6696e903ca5a5d6d1dfc775
2019-05-03 02:32:44.643591 7fea2734b700 10 generated signature = 8d80dab573fb61c2eb49a9b36f817199bd10d4504804a230a7273b963699a215
2019-05-03 02:32:44.643593 7fea2734b700 15 string_to_sign=AWS4-HMAC-SHA256
20190503T063244Z
20190503/us-east-1/s3/aws4_request
17c64c14c926ae9de43f0b0647eb5314e7832d71c1b71af21917ba8e8e524dad
2019-05-03 02:32:44.643607 7fea2734b700 15 server signature=8d80dab573fb61c2eb49a9b36f817199bd10d4504804a230a7273b963699a215
2019-05-03 02:32:44.643617 7fea2734b700 15 client signature=8d80dab573fb61c2eb49a9b36f817199bd10d4504804a230a7273b963699a215
2019-05-03 02:32:44.643618 7fea2734b700 15 compare=0
2019-05-03 02:32:44.643641 7fea2734b700 20 rgw::auth::s3::LocalEngine granted access
2019-05-03 02:32:44.643643 7fea2734b700 20 rgw::auth::s3::AWSAuthStrategy granted access
2019-05-03 02:32:44.643661 7fea2734b700 2 req 90:0.001347:s3:GET /BUCKET:get_bucket_versioning:normalizing buckets and tenants
2019-05-03 02:32:44.643682 7fea2734b700 10 s->object=<NULL> s->bucket=BUCKET
2019-05-03 02:32:44.643713 7fea2734b700 2 req 90:0.001398:s3:GET /BUCKET:get_bucket_versioning:init permissions
2019-05-03 02:32:44.643890 7fea2734b700 20 get_system_obj_state: rctx=0x7fea273439d0 obj=us-east.rgw.meta:root:BUCKET state=0x55a5056a2860 s->prefetch_data=0
2019-05-03 02:32:44.643992 7fea2734b700 10 cache get: name=us-east.rgw.meta+root+BUCKET : type miss (requested=0x16, cached=0x0)
2019-05-03 02:32:44.644404 7fea2734b700 1 -- 10.1.1.0:0/700479603 --> 10.1.1.2:6804/196285 -- osd_op(unknown.0.0:372251 6.0 6:0c3a7a08:root::BUCKET:head [call version.read,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e504) v8 -- 0x55a50450b8c0 con 0
2019-05-03 02:32:44.647587 7fea71ca1700 1 -- 10.1.1.0:0/700479603 <== osd.0 10.1.1.2:6804/196285 138773 ==== osd_op_reply(372251 BUCKET [call,getxattrs,stat] v0'0 uv0 ondisk = 2 ((2) No such file or directory)) v8 ==== 234+0+0 (696673678 0 0) 0x55a50450b8c0 con 0x55a5038cc000
2019-05-03 02:32:44.648616 7fea2734b700 10 cache put: name=us-east.rgw.meta+root+BUCKET info.flags=0x0
2019-05-03 02:32:44.648661 7fea2734b700 10 moving us-east.rgw.meta+root+BUCKET to cache LRU end
2019-05-03 02:32:44.648759 7fea2734b700 20 get_system_obj_state: rctx=0x7fea27343610 obj=us-east.rgw.meta:users.uid:manju state=0x55a5056a2a00 s>prefetch_data=0
2019-05-03 02:32:44.648819 7fea2734b700 10 cache get: name=us-east.rgw.meta+users.uid+manju : hit (requested=0x6, cached=0x17)
2019-05-03 02:32:44.648864 7fea2734b700 20 get_system_obj_state: s->obj_tag was set empty
2019-05-03 02:32:44.648881 7fea2734b700 20 Read xattr: user.rgw.idtag
2019-05-03 02:32:44.648964 7fea2734b700 2 req 90:0.006647:s3:GET /BUCKET:get_bucket_versioning:recalculating target
2019-05-03 02:32:44.648970 7fea2734b700 2 req 90:0.006656:s3:GET /BUCKET:get_bucket_versioning:reading permissions
2019-05-03 02:32:44.648991 7fea2734b700 2 req 90:0.006676:s3:GET /BUCKET:get_bucket_versioning:init op
2019-05-03 02:32:44.649007 7fea2734b700 2 req 90:0.006692:s3:GET /BUCKET:get_bucket_versioning:verifying op mask
2019-05-03 02:32:44.649031 7fea2734b700 20 required_mask= 1 user.op_mask=7
2019-05-03 02:32:44.649037 7fea2734b700 2 req 90:0.006722:s3:GET /BUCKET:get_bucket_versioning:verifying op permissions
2019-05-03 02:32:44.649056 7fea2734b700 2 req 90:0.006741:s3:GET /BUCKET:get_bucket_versioning:verifying op params
2019-05-03 02:32:44.649070 7fea2734b700 2 req 90:0.006745:s3:GET /BUCKET:get_bucket_versioning:pre-executing
2019-05-03 02:32:44.649083 7fea2734b700 2 req 90:0.006769:s3:GET /BUCKET:get_bucket_versioning:executing
2019-05-03 02:32:44.649088 7fea2734b700 2 req 90:0.006773:s3:GET /BUCKET:get_bucket_versioning:completing
2019-05-03 02:32:44.649366 7fea2734b700 2 req 90:0.007050:s3:GET /BUCKET:get_bucket_versioning:op status=0
2019-05-03 02:32:44.649371 7fea2734b700 2 req 90:0.007056:s3:GET /BUCKET:get_bucket_versioning:http status=200
2019-05-03 02:32:44.649382 7fea2734b700 1 ====== req done req=0x7fea27344f90 op status=0 http_status=200 ======

Actions

Copy link

Updated by Prashant D almost 5 years ago

Looks like tracker 38420 addresses this issue, with latest master could not reproduce this issue :

$ git log cf66a6d5a7e
commit cf66a6d5a7eee294313a1a08d0524daf939747e5
Author: Abhishek Lekshmanan <abhishek@suse.com>
Date: Thu Feb 21 17:06:52 2019 +0100

rgw: return ERR_NO_SUCH_BUCKET early while evaluating bucket policy

Right now we create a ERR_NO_SUCH_BUCKET ret code but continue further
    processing. Since this ret code isn't returned at any stage we end up creating a
    bucket instance anyway which shouldn't happen and then succeeding the client
    call in cases like put bucket versioning. Return an error code early in these
    cases

Fixes: http://tracker.ceph.com/issues/38420
    Signed-off-by: Abhishek Lekshmanan &lt;abhishek@suse.com&gt;

Actions

Copy link

Updated by Prashant D almost 5 years ago

Assignee deleted (~~Prashant D~~)

Actions

Copy link

Updated by Casey Bodley almost 5 years ago

Is duplicate of Bug #38420: rgw: success returned for put bucket versioning on a non existant bucket added

Actions

Copy link

Updated by Casey Bodley almost 5 years ago

Status changed from New to Duplicate

will get backports for http://tracker.ceph.com/issues/38420

Actions

Copy link

Also available in: Atom PDF

Project

General

Profile

Ceph » rgw

Custom queries

Bug #39622

rgw:RGWGetBucketVersioning on non-existing bucket does not return NoSuchBucket (http=404)

Updated by Prashant D almost 5 years ago

Updated by Prashant D almost 5 years ago

Updated by Prashant D almost 5 years ago

Updated by Prashant D almost 5 years ago

Updated by Casey Bodley almost 5 years ago

Updated by Casey Bodley almost 5 years ago