Bug #16463
closedAWS4 Presigned URL not accepted by RGW
0%
Description
I have a 3 node cluster (3 mons, 3 rgw, 36 osd) behind a haproxy instance.
I try to use mc (https://github.com/minio/mc) to generate a presigned GET request for an object. However the object is not retrieved from RGW instead it results in a 403 Forbidden.
A signed v2 URL works.
The mc command to reproduce the request are:
mc config host add rgw https://<endpoint> <access_key> <secret_key> S3v4
mc share download rgw/<bucket>/<key>
It will output the signed URL to the console:
URL: https://***REDACTED***/test3/201623662.pdf
Expire: 7 days 0 hours 0 minutes 0 seconds
Share: https://***REDACTED***/test3/201623662.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=***REDACTED***%2F20160623%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20160623T210052Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=c6a1722018017f3c1a9c2fbedb0edd7a49b1fbc0d6e6db7fc02dd31f150d1469
If you use wget to receive the URL the following output occurs:
...
HTTP request sent, awaiting response... 403 Forbidden
2016-06-23 23:02:08 ERROR 403: Forbidden.
The debug rgw 20 log shows the following entries:
2016-06-23 21:02:58.903975 7f9626ffd700 20 RGWEnv::set(): HTTP_USER_AGENT: Wget/1.18 (darwin15.5.0)
2016-06-23 21:02:58.903992 7f9626ffd700 20 RGWEnv::set(): HTTP_ACCEPT: /*
2016-06-23 21:02:58.903996 7f9626ffd700 20 RGWEnv::set(): HTTP_ACCEPT_ENCODING: identity
2016-06-23 21:02:58.904001 7f9626ffd700 20 RGWEnv::set(): HTTP_HOST: *REDACTED
2016-06-23 21:02:58.904003 7f9626ffd700 20 RGWEnv::set(): HTTP_X_FORWARDED_FOR: REDACTED
2016-06-23 21:02:58.904005 7f9626ffd700 20 RGWEnv::set(): HTTP_CONNECTION: close
2016-06-23 21:02:58.904008 7f9626ffd700 20 RGWEnv::set(): REQUEST_METHOD: GET
2016-06-23 21:02:58.904010 7f9626ffd700 20 RGWEnv::set(): REQUEST_URI: /test3/201623662.pdf
2016-06-23 21:02:58.904017 7f9626ffd700 20 RGWEnv::set(): QUERY_STRING: X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=***REDACTED***%2F20160623%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20160623T210052Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=c6a1722018017f3c1a9c2fbedb0edd7a49b1fbc0d6e6db7fc02dd31f150d1469
2016-06-23 21:02:58.904024 7f9626ffd700 20 RGWEnv::set(): REMOTE_USER:
2016-06-23 21:02:58.904026 7f9626ffd700 20 RGWEnv::set(): SCRIPT_URI: /test3/201623662.pdf
2016-06-23 21:02:58.904029 7f9626ffd700 20 RGWEnv::set(): SERVER_PORT: 7480
2016-06-23 21:02:58.904030 7f9626ffd700 20 HTTP_ACCEPT=*/*
2016-06-23 21:02:58.904032 7f9626ffd700 20 HTTP_ACCEPT_ENCODING=identity
2016-06-23 21:02:58.904033 7f9626ffd700 20 HTTP_CONNECTION=close
2016-06-23 21:02:58.904034 7f9626ffd700 20 HTTP_HOST=***REDACTED***
2016-06-23 21:02:58.904036 7f9626ffd700 20 HTTP_USER_AGENT=Wget/1.18 (darwin15.5.0)
2016-06-23 21:02:58.904037 7f9626ffd700 20 HTTP_X_FORWARDED_FOR=***REDACTED***
2016-06-23 21:02:58.904038 7f9626ffd700 20 QUERY_STRING=X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=***REDACTED***%2F20160623%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20160623T210052Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=c6a1722018017f3c1a9c2fbedb0edd7a49b1fbc0d6e6db7fc02dd31f150d1469
2016-06-23 21:02:58.904045 7f9626ffd700 20 REMOTE_USER=
2016-06-23 21:02:58.904046 7f9626ffd700 20 REQUEST_METHOD=GET
2016-06-23 21:02:58.904047 7f9626ffd700 20 REQUEST_URI=/test3/201623662.pdf
2016-06-23 21:02:58.904048 7f9626ffd700 20 SCRIPT_URI=/test3/201623662.pdf
2016-06-23 21:02:58.904049 7f9626ffd700 20 SERVER_PORT=7480
2016-06-23 21:02:58.904052 7f9626ffd700 1 ====== starting new request req=0x7f9626ff7710 =====
2016-06-23 21:02:58.904076 7f9626ffd700 2 req 49:0.000024::GET /test3/201623662.pdf::initializing for trans_id = tx000000000000000000031-00576c4e82-58e5a-default
2016-06-23 21:02:58.904082 7f9626ffd700 10 host=***REDACTED***
2016-06-23 21:02:58.904089 7f9626ffd700 20 subdomain= domain=***REDACTED*** in_hosted_domain=1 in_hosted_domain_s3website=0
2016-06-23 21:02:58.904148 7f9626ffd700 20 get_handler handler=22RGWHandler_REST_Obj_S3
2016-06-23 21:02:58.904154 7f9626ffd700 10 handler=22RGWHandler_REST_Obj_S3
2016-06-23 21:02:58.904156 7f9626ffd700 2 req 49:0.000104:s3:GET /test3/201623662.pdf::getting op 0
2016-06-23 21:02:58.904162 7f9626ffd700 10 op=21RGWGetObj_ObjStore_S3
2016-06-23 21:02:58.904164 7f9626ffd700 2 req 49:0.000112:s3:GET /test3/201623662.pdf:get_obj:authorizing
2016-06-23 21:02:58.904205 7f9626ffd700 10 v4 credential format = REDACTED/20160623/us-east-1/s3/aws4_request
2016-06-23 21:02:58.904209 7f9626ffd700 10 access key id = REDACTED
2016-06-23 21:02:58.904211 7f9626ffd700 10 credential scope = 20160623/us-east-1/s3/aws4_request
2016-06-23 21:02:58.904281 7f9626ffd700 10 canonical headers format = host:***REDACTED***:7480
2016-06-23 21:02:58.904285 7f9626ffd700 10 payload request hash = UNSIGNED-PAYLOAD
2016-06-23 21:02:58.904342 7f9626ffd700 10 canonical request = GET
/test3/201623662.pdf
X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=***REDACTED***%2F20160623%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20160623T210052Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host
host:***REDACTED***:7480
host
UNSIGNED-PAYLOAD
2016-06-23 21:02:58.904344 7f9626ffd700 10 canonical request hash = 83a855651c9a6ff01f51bea38951fa43c7491b816f24d0f11cf8e92478c2ee2d
2016-06-23 21:02:58.904349 7f9626ffd700 10 string to sign = AWS4-HMAC-SHA256
20160623T210052Z
20160623/us-east-1/s3/aws4_request
83a855651c9a6ff01f51bea38951fa43c7491b816f24d0f11cf8e92478c2ee2d
2016-06-23 21:02:58.904412 7f9626ffd700 10 date_k = f5abcff2800eaff43c902bea85570d14d9eebf0dbcd956fd5209abcd6eaec1fe
2016-06-23 21:02:58.904451 7f9626ffd700 10 region_k = 61a870b01101b1bb02b82dac0b2b9b04c695ecbd26c5dbe733845e0a89676106
2016-06-23 21:02:58.904485 7f9626ffd700 10 service_k = 2c8cdecb7b9e65c6f2b549dbb1c04d42a2a01dc801656533fc79c12f525bcc4a
2016-06-23 21:02:58.904518 7f9626ffd700 10 signing_k = 82dfdfdec4dcdaaff20417ca180323851422f7afabca3d3a31a3d0f773209b1a
2016-06-23 21:02:58.904554 7f9626ffd700 10 signature_k = 00a40c296701da605f09b10aab4ae7c3f1f9aec4dc7b0a48fd22c7285dd38b29
2016-06-23 21:02:58.904559 7f9626ffd700 10 new signature = 00a40c296701da605f09b10aab4ae7c3f1f9aec4dc7b0a48fd22c7285dd38b29
2016-06-23 21:02:58.904560 7f9626ffd700 10 ----------------------------- Verifying signatures
2016-06-23 21:02:58.904561 7f9626ffd700 10 Signature = c6a1722018017f3c1a9c2fbedb0edd7a49b1fbc0d6e6db7fc02dd31f150d1469
2016-06-23 21:02:58.904565 7f9626ffd700 10 New Signature = 00a40c296701da605f09b10aab4ae7c3f1f9aec4dc7b0a48fd22c7285dd38b29
2016-06-23 21:02:58.904566 7f9626ffd700 10 -----------------------------
2016-06-23 21:02:58.904571 7f9626ffd700 10 failed to authorize request
2016-06-23 21:02:58.904573 7f9626ffd700 20 handler->ERRORHANDLER: err_no=-2027 new_err_no=-2027
2016-06-23 21:02:58.904705 7f9626ffd700 2 req 49:0.000653:s3:GET /test3/201623662.pdf:get_obj:op status=0
2016-06-23 21:02:58.904714 7f9626ffd700 2 req 49:0.000662:s3:GET /test3/201623662.pdf:get_obj:http status=403
2016-06-23 21:02:58.904719 7f9626ffd700 1 ====== req done req=0x7f9626ff7710 op status=0 http_status=403 ======
2016-06-23 21:02:58.904733 7f9626ffd700 20 process_request() returned -2027
2016-06-23 21:02:58.904766 7f9626ffd700 1 civetweb: 0x7f968c000bb0: REDACTED - - [23/Jun/2016:21:02:58 +0000] "GET /test3/201623662.pdf HTTP/1.1" 403 0 - Wget/1.18 (darwin15.5.0)
2016-06-23 21:02:59.586329 7f97b37fe700 2 RGWDataChangesLog::ChangesRenewThread: start