Ceph » rgw

This is a part of issues discussed on ceph-devel ML: http://marc.info/?t=148671813300008
It is related to #19043 and #19043.

Amazon S3 have a feature to redirect the request if a request arrived at wrong endpoint.
http://docs.aws.amazon.com/AmazonS3/latest/dev/Redirects.html

On current RGW code, it behaves similar but incomplete. (RGW v10.2.5)

When a request for secondary zonegroup is arrived at master zonegroup, it returns "301 Moved Permanently" but no "Location" header is included. Thus, client can't redirect to correct location.

# s3cmd -c s3jp-east.cfg -d get s3://bucket2/10m 10m2
DEBUG: s3cmd version 1.6.1
DEBUG: ConfigParser: Reading file 's3jp-east.cfg'
...
DEBUG: CreateRequest: resource[uri]=/10m
DEBUG: Using signature v4
DEBUG: get_hostname(bucket2): node5:8080
DEBUG: canonical_headers = host:node5:8080
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20170222T054257Z

DEBUG: Canonical Request:
GET
/bucket2/10m

host:node5:8080
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20170222T054257Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
----------------------
DEBUG: signature-v4 headers: {'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': 'AWS4-HMAC-SHA256 Credential=F5JX1TL9B6E78LQDSYCM/20170222/jp/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=481ee78c98bf3ea4740bbb58cbbf4cb1e98b1da2d426eabfd79e6b42dbb34bcc', 'x-amz-date': '20170222T054257Z'}
DEBUG: Unicodising '10m2' using UTF-8
download: 's3://bucket2/10m' -> '10m2'  [1 of 1]
DEBUG: get_hostname(bucket2): node5:8080
DEBUG: ConnMan.get(): creating new connection: http://node5:8080
DEBUG: non-proxied HTTPConnection(node5:8080)
DEBUG: format_uri(): /bucket2/10m
DEBUG: Response: {'status': 301, 'headers': {'date': 'Wed, 22 Feb 2017 05:42:57 GMT', 'content-length': '0', 'x-amz-request-id': 'tx00000000000000000000a-0058ad24e1-151c-jp-east'}, 'reason': 'Moved Permanently'}
DEBUG: S3Error: 301 (Moved Permanently)
DEBUG: HttpHeader: date: Wed, 22 Feb 2017 05:42:57 GMT
DEBUG: HttpHeader: content-length: 0
DEBUG: HttpHeader: x-amz-request-id: tx00000000000000000000a-0058ad24e1-151c-jp-east
DEBUG: object_get failed for '10m2', deleting...
DEBUG: DeUnicodising u'10m2' using UTF-8
ERROR: S3 error: 301 (Moved Permanently)

To support this feature, some modifications are required:

• Synchronize bucket metadata (at least location information) to secondary zonegroups.
  Currently, bucket metadata is collected in the master zonegroup, but it is not delivered to secondary zonegroups. A secondary zonegroup doesn't have bucket location information for other zonegroups.
• Respond "307 Temporary Redirect" instead of "301 Moved Permanently"?
  We may not follow Amazon S3 spec so strictly.
• Present correct URL on Location header.
  I think "hostnames" field of zonegroup should be used to locate correct URL for the other zonegroup. It is similar to s3website feature.

{
    "id": "b9fb59d2-7c90-4cfd-a5b4-006adbda2af9",
    "name": "default",
    "api_name": "east",
    "is_master": "false",
    "endpoints": [
        "http:\/\/node5:80" 
    ],
    "hostnames": [
        "USE-THIS-FIELD.domain.com" 
    ],
    "hostnames_s3website": [],
    "master_zone": "6216eff2-1146-4ba1-9368-43d70dd45975",
    "zones": [
        {
            "id": "6216eff2-1146-4ba1-9368-43d70dd45975",
            "name": "default",
            "endpoints": [
                "http:\/\/node5:80" 
            ],
            "log_meta": "false",
            "log_data": "false",
            "bucket_index_max_shards": 0,
            "read_only": "false" 
        }
    ],
    "placement_targets": [
        {
            "name": "default-placement",
            "tags": []
        }
    ],
    "default_placement": "default-placement",
    "realm_id": "d2def91a-84c6-4383-84df-c2b30bf91bab" 
}