Actions
Feature #19052
openrgw: multiple zonegroups: support of request redirection to access different zonegroup
Status:
New
Priority:
Normal
Assignee:
-
Target version:
-
% Done:
0%
Source:
Tags:
Backport:
Reviewed:
Affected Versions:
Pull request ID:
Description
This is a part of issues discussed on ceph-devel ML: http://marc.info/?t=148671813300008
It is related to #19043 and #19043.
Amazon S3 have a feature to redirect the request if a request arrived at wrong endpoint.
http://docs.aws.amazon.com/AmazonS3/latest/dev/Redirects.html
On current RGW code, it behaves similar but incomplete. (RGW v10.2.5)
When a request for secondary zonegroup is arrived at master zonegroup, it returns "301 Moved Permanently" but no "Location" header is included. Thus, client can't redirect to correct location.
# s3cmd -c s3jp-east.cfg -d get s3://bucket2/10m 10m2 DEBUG: s3cmd version 1.6.1 DEBUG: ConfigParser: Reading file 's3jp-east.cfg' ... DEBUG: CreateRequest: resource[uri]=/10m DEBUG: Using signature v4 DEBUG: get_hostname(bucket2): node5:8080 DEBUG: canonical_headers = host:node5:8080 x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 x-amz-date:20170222T054257Z DEBUG: Canonical Request: GET /bucket2/10m host:node5:8080 x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 x-amz-date:20170222T054257Z host;x-amz-content-sha256;x-amz-date e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 ---------------------- DEBUG: signature-v4 headers: {'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': 'AWS4-HMAC-SHA256 Credential=F5JX1TL9B6E78LQDSYCM/20170222/jp/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=481ee78c98bf3ea4740bbb58cbbf4cb1e98b1da2d426eabfd79e6b42dbb34bcc', 'x-amz-date': '20170222T054257Z'} DEBUG: Unicodising '10m2' using UTF-8 download: 's3://bucket2/10m' -> '10m2' [1 of 1] DEBUG: get_hostname(bucket2): node5:8080 DEBUG: ConnMan.get(): creating new connection: http://node5:8080 DEBUG: non-proxied HTTPConnection(node5:8080) DEBUG: format_uri(): /bucket2/10m DEBUG: Response: {'status': 301, 'headers': {'date': 'Wed, 22 Feb 2017 05:42:57 GMT', 'content-length': '0', 'x-amz-request-id': 'tx00000000000000000000a-0058ad24e1-151c-jp-east'}, 'reason': 'Moved Permanently'} DEBUG: S3Error: 301 (Moved Permanently) DEBUG: HttpHeader: date: Wed, 22 Feb 2017 05:42:57 GMT DEBUG: HttpHeader: content-length: 0 DEBUG: HttpHeader: x-amz-request-id: tx00000000000000000000a-0058ad24e1-151c-jp-east DEBUG: object_get failed for '10m2', deleting... DEBUG: DeUnicodising u'10m2' using UTF-8 ERROR: S3 error: 301 (Moved Permanently)
To support this feature, some modifications are required:
- Synchronize bucket metadata (at least location information) to secondary zonegroups.
Currently, bucket metadata is collected in the master zonegroup, but it is not delivered to secondary zonegroups. A secondary zonegroup doesn't have bucket location information for other zonegroups. - Respond
"307 Temporary Redirect"
instead of"301 Moved Permanently"
?
We may not follow Amazon S3 spec so strictly. - Present correct URL on Location header.
I think"hostnames"
field of zonegroup should be used to locate correct URL for the other zonegroup. It is similar to s3website feature.
{
"id": "b9fb59d2-7c90-4cfd-a5b4-006adbda2af9",
"name": "default",
"api_name": "east",
"is_master": "false",
"endpoints": [
"http:\/\/node5:80"
],
"hostnames": [
"USE-THIS-FIELD.domain.com"
],
"hostnames_s3website": [],
"master_zone": "6216eff2-1146-4ba1-9368-43d70dd45975",
"zones": [
{
"id": "6216eff2-1146-4ba1-9368-43d70dd45975",
"name": "default",
"endpoints": [
"http:\/\/node5:80"
],
"log_meta": "false",
"log_data": "false",
"bucket_index_max_shards": 0,
"read_only": "false"
}
],
"placement_targets": [
{
"name": "default-placement",
"tags": []
}
],
"default_placement": "default-placement",
"realm_id": "d2def91a-84c6-4383-84df-c2b30bf91bab"
}
Files
Actions