Project

General

Profile

Bug #15501

multisite: rgw returns ErrorInvalidArgument when creating buckets when master zone is down

Added by Abhishek Lekshmanan almost 8 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
Normal
Target version:
-
% Done:

0%

Source:
Community (dev)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

When master zone is down, bucket creation fails, however RGW returns -EINVALID, which appears as InvalidArgument to the client, we should probably return 503 ServiceUnavailable instead

# s3 -us create foobar2

ERROR: ErrorInvalidArgument
  Extra Details:
    BucketName: foobar2
    RequestId: tx000000000000000000002-00570fb9fc-101c-us-west
    HostId: 101c-us-west-us

RGW logs

6-04-14 17:13:36.952705 7fb07b7f6700 10 handler=25RGWHandler_REST_Bucket_S3
2016-04-14 17:13:36.952707 7fb07b7f6700  2 req 2169:0.000065:s3:PUT /foobar2/::getting op 1
2016-04-14 17:13:36.952713 7fb07b7f6700 10 op=27RGWCreateBucket_ObjStore_S3
2016-04-14 17:13:36.952715 7fb07b7f6700  2 req 2169:0.000073:s3:PUT /foobar2/:create_bucket:authorizing
2016-04-14 17:13:36.952748 7fb07b7f6700 10 get_canon_resource(): dest=/foobar2/
2016-04-14 17:13:36.952750 7fb07b7f6700 10 auth_hdr:
PUT

x-amz-date:Thu, 14 Apr 2016 15:13:36 GMT
/foobar2/
2016-04-14 17:13:36.952806 7fb07b7f6700 15 calculated digest=FqbDvRiZPAQmuScbVYOK1b685Jk=
2016-04-14 17:13:36.952808 7fb07b7f6700 15 auth_sign=FqbDvRiZPAQmuScbVYOK1b685Jk=
2016-04-14 17:13:36.952809 7fb07b7f6700 15 compare=0
2016-04-14 17:13:36.952812 7fb07b7f6700  2 req 2169:0.000171:s3:PUT /foobar2/:create_bucket:normalizing buckets and tenants
2016-04-14 17:13:36.952816 7fb07b7f6700 10 s->object=<NULL> s->bucket=foobar2
2016-04-14 17:13:36.952818 7fb07b7f6700  2 req 2169:0.000177:s3:PUT /foobar2/:create_bucket:init permissions
2016-04-14 17:13:36.952820 7fb07b7f6700  2 req 2169:0.000179:s3:PUT /foobar2/:create_bucket:recalculating target
2016-04-14 17:13:36.952823 7fb07b7f6700  2 req 2169:0.000181:s3:PUT /foobar2/:create_bucket:reading permissions
2016-04-14 17:13:36.952825 7fb07b7f6700  2 req 2169:0.000184:s3:PUT /foobar2/:create_bucket:init op
2016-04-14 17:13:36.952827 7fb07b7f6700  2 req 2169:0.000186:s3:PUT /foobar2/:create_bucket:verifying op mask
2016-04-14 17:13:36.952830 7fb07b7f6700 20 required_mask= 2 user.op_mask=7
2016-04-14 17:13:36.952831 7fb07b7f6700  2 req 2169:0.000190:s3:PUT /foobar2/:create_bucket:verifying op permissions
2016-04-14 17:13:36.953486 7fb07b7f6700  2 req 2169:0.000844:s3:PUT /foobar2/:create_bucket:verifying op params
2016-04-14 17:13:36.953492 7fb07b7f6700  2 req 2169:0.000851:s3:PUT /foobar2/:create_bucket:pre-executing
2016-04-14 17:13:36.953518 7fb07b7f6700  2 req 2169:0.000876:s3:PUT /foobar2/:create_bucket:executing
2016-04-14 17:13:36.953556 7fb07b7f6700 20 get_system_obj_state: rctx=0x7fb07b7ef5f0 obj=us-west.rgw.data.root:foobar2 state=0x7fb0b8031018 s->prefetch_data=0
2016-04-14 17:13:36.953567 7fb07b7f6700 10 cache get: name=us-west.rgw.data.root+foobar2 : type miss (requested=22, cached=0)
2016-04-14 17:13:36.954070 7fb07b7f6700 10 cache put: name=us-west.rgw.data.root+foobar2 info.flags=0
2016-04-14 17:13:36.954076 7fb07b7f6700 10 moving us-west.rgw.data.root+foobar2 to cache LRU end
2016-04-14 17:13:36.954084 7fb07b7f6700  0 sending create_bucket request to master zonegroup
2016-04-14 17:13:36.954102 7fb07b7f6700 20 RGWEnv::set(): HTTP_DATE: Thu Apr 14 15:13:36 2016
2016-04-14 17:13:36.954106 7fb07b7f6700 20 > HTTP_DATE -> Thu Apr 14 15:13:36 2016
2016-04-14 17:13:36.954113 7fb07b7f6700 10 get_canon_resource(): dest=/foobar2/
2016-04-14 17:13:36.954115 7fb07b7f6700 10 generated canonical header: PUT

Thu Apr 14 15:13:36 2016
/foobar2/
2016-04-14 17:13:36.954148 7fb07b7f6700 15 generated auth header: AWS 1555b35654ad1656d804:dMi1/LnQobSwA2ZW+jE0uow/Oxo=
2016-04-14 17:13:36.954173 7fb07b7f6700 20 sending request to http://localhost:8001/foobar2/?&rgwx-uid=foo&rgwx-region=1df6de86-12ea-4706-8d5c-68860b94af1e
2016-04-14 17:13:36.958376 7fb07b7f6700  0 curl_easy_perform returned error: Failed to connect to localhost port 8001: Connection refused
2016-04-14 17:13:36.958416 7fb07b7f6700  2 req 2169:0.005774:s3:PUT /foobar2/:create_bucket:completing
2016-04-14 17:13:36.958464 7fb07b7f6700  2 req 2169:0.005822:s3:PUT /foobar2/:create_bucket:op status=-22
2016-04-14 17:13:36.958466 7fb07b7f6700  2 req 2169:0.005825:s3:PUT /foobar2/:create_bucket:http status=400
2016-04-14 17:13:36.958469 7fb07b7f6700  1 ====== req done req=0x7fb07b7f0710 op status=-22 http_status=400 ======

EDIT format fixing

History

#1 Updated by Abhishek Lekshmanan almost 8 years ago

  • Description updated (diff)

#2 Updated by Abhishek Lekshmanan almost 8 years ago

  • Description updated (diff)

#3 Updated by Abhishek Lekshmanan almost 8 years ago

  • Status changed from New to In Progress
  • Assignee set to Abhishek Lekshmanan

#4 Updated by Abhishek Lekshmanan almost 8 years ago

  • Status changed from In Progress to Fix Under Review

#5 Updated by Abhishek Lekshmanan over 6 years ago

  • Status changed from Fix Under Review to Resolved

Also available in: Atom PDF