Bug #15501
multisite: rgw returns ErrorInvalidArgument when creating buckets when master zone is down
% Done:
0%
Source:
Community (dev)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
When master zone is down, bucket creation fails, however RGW returns -EINVALID, which appears as InvalidArgument to the client, we should probably return 503 ServiceUnavailable instead
# s3 -us create foobar2 ERROR: ErrorInvalidArgument Extra Details: BucketName: foobar2 RequestId: tx000000000000000000002-00570fb9fc-101c-us-west HostId: 101c-us-west-us
RGW logs
6-04-14 17:13:36.952705 7fb07b7f6700 10 handler=25RGWHandler_REST_Bucket_S3 2016-04-14 17:13:36.952707 7fb07b7f6700 2 req 2169:0.000065:s3:PUT /foobar2/::getting op 1 2016-04-14 17:13:36.952713 7fb07b7f6700 10 op=27RGWCreateBucket_ObjStore_S3 2016-04-14 17:13:36.952715 7fb07b7f6700 2 req 2169:0.000073:s3:PUT /foobar2/:create_bucket:authorizing 2016-04-14 17:13:36.952748 7fb07b7f6700 10 get_canon_resource(): dest=/foobar2/ 2016-04-14 17:13:36.952750 7fb07b7f6700 10 auth_hdr: PUT x-amz-date:Thu, 14 Apr 2016 15:13:36 GMT /foobar2/ 2016-04-14 17:13:36.952806 7fb07b7f6700 15 calculated digest=FqbDvRiZPAQmuScbVYOK1b685Jk= 2016-04-14 17:13:36.952808 7fb07b7f6700 15 auth_sign=FqbDvRiZPAQmuScbVYOK1b685Jk= 2016-04-14 17:13:36.952809 7fb07b7f6700 15 compare=0 2016-04-14 17:13:36.952812 7fb07b7f6700 2 req 2169:0.000171:s3:PUT /foobar2/:create_bucket:normalizing buckets and tenants 2016-04-14 17:13:36.952816 7fb07b7f6700 10 s->object=<NULL> s->bucket=foobar2 2016-04-14 17:13:36.952818 7fb07b7f6700 2 req 2169:0.000177:s3:PUT /foobar2/:create_bucket:init permissions 2016-04-14 17:13:36.952820 7fb07b7f6700 2 req 2169:0.000179:s3:PUT /foobar2/:create_bucket:recalculating target 2016-04-14 17:13:36.952823 7fb07b7f6700 2 req 2169:0.000181:s3:PUT /foobar2/:create_bucket:reading permissions 2016-04-14 17:13:36.952825 7fb07b7f6700 2 req 2169:0.000184:s3:PUT /foobar2/:create_bucket:init op 2016-04-14 17:13:36.952827 7fb07b7f6700 2 req 2169:0.000186:s3:PUT /foobar2/:create_bucket:verifying op mask 2016-04-14 17:13:36.952830 7fb07b7f6700 20 required_mask= 2 user.op_mask=7 2016-04-14 17:13:36.952831 7fb07b7f6700 2 req 2169:0.000190:s3:PUT /foobar2/:create_bucket:verifying op permissions 2016-04-14 17:13:36.953486 7fb07b7f6700 2 req 2169:0.000844:s3:PUT /foobar2/:create_bucket:verifying op params 2016-04-14 17:13:36.953492 7fb07b7f6700 2 req 2169:0.000851:s3:PUT /foobar2/:create_bucket:pre-executing 2016-04-14 17:13:36.953518 7fb07b7f6700 2 req 2169:0.000876:s3:PUT /foobar2/:create_bucket:executing 2016-04-14 17:13:36.953556 7fb07b7f6700 20 get_system_obj_state: rctx=0x7fb07b7ef5f0 obj=us-west.rgw.data.root:foobar2 state=0x7fb0b8031018 s->prefetch_data=0 2016-04-14 17:13:36.953567 7fb07b7f6700 10 cache get: name=us-west.rgw.data.root+foobar2 : type miss (requested=22, cached=0) 2016-04-14 17:13:36.954070 7fb07b7f6700 10 cache put: name=us-west.rgw.data.root+foobar2 info.flags=0 2016-04-14 17:13:36.954076 7fb07b7f6700 10 moving us-west.rgw.data.root+foobar2 to cache LRU end 2016-04-14 17:13:36.954084 7fb07b7f6700 0 sending create_bucket request to master zonegroup 2016-04-14 17:13:36.954102 7fb07b7f6700 20 RGWEnv::set(): HTTP_DATE: Thu Apr 14 15:13:36 2016 2016-04-14 17:13:36.954106 7fb07b7f6700 20 > HTTP_DATE -> Thu Apr 14 15:13:36 2016 2016-04-14 17:13:36.954113 7fb07b7f6700 10 get_canon_resource(): dest=/foobar2/ 2016-04-14 17:13:36.954115 7fb07b7f6700 10 generated canonical header: PUT Thu Apr 14 15:13:36 2016 /foobar2/ 2016-04-14 17:13:36.954148 7fb07b7f6700 15 generated auth header: AWS 1555b35654ad1656d804:dMi1/LnQobSwA2ZW+jE0uow/Oxo= 2016-04-14 17:13:36.954173 7fb07b7f6700 20 sending request to http://localhost:8001/foobar2/?&rgwx-uid=foo&rgwx-region=1df6de86-12ea-4706-8d5c-68860b94af1e 2016-04-14 17:13:36.958376 7fb07b7f6700 0 curl_easy_perform returned error: Failed to connect to localhost port 8001: Connection refused 2016-04-14 17:13:36.958416 7fb07b7f6700 2 req 2169:0.005774:s3:PUT /foobar2/:create_bucket:completing 2016-04-14 17:13:36.958464 7fb07b7f6700 2 req 2169:0.005822:s3:PUT /foobar2/:create_bucket:op status=-22 2016-04-14 17:13:36.958466 7fb07b7f6700 2 req 2169:0.005825:s3:PUT /foobar2/:create_bucket:http status=400 2016-04-14 17:13:36.958469 7fb07b7f6700 1 ====== req done req=0x7fb07b7f0710 op status=-22 http_status=400 ======
EDIT format fixing
History
#1 Updated by Abhishek Lekshmanan almost 8 years ago
- Description updated (diff)
#2 Updated by Abhishek Lekshmanan almost 8 years ago
- Description updated (diff)
#3 Updated by Abhishek Lekshmanan almost 8 years ago
- Status changed from New to In Progress
- Assignee set to Abhishek Lekshmanan
#4 Updated by Abhishek Lekshmanan almost 8 years ago
- Status changed from In Progress to Fix Under Review
#5 Updated by Abhishek Lekshmanan over 6 years ago
- Status changed from Fix Under Review to Resolved