Ceph » rgw

s3cmd uses v4 signatures by default, and generates them differently than RGW does. I'd trust s3cmd at this point, because it works with AWS...

RGW is building the canonical URL incorrectly.

s3cmd:

# s3cmd -c s3cfg-rgw ls s3://pewpew --debug
DEBUG: Unicodising 's3://pewpew' using UTF-8
DEBUG: Command: ls
DEBUG: Bucket 's3://pewpew':
DEBUG: CreateRequest: resource[uri]=/
DEBUG: Using signature v4
DEBUG: get_hostname(pewpew): pewpew.CENSORED
DEBUG: canonical_headers = host:pewpew.CENSORED
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20160404T214839Z

DEBUG: Canonical Request:
GET
/
delimiter=%2F
host:pewpew.CENSORED
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20160404T214839Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
----------------------
DEBUG: signature-v4 headers: {'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': 'AWS4-HMAC-SHA256 Credential=JZXZ88D0YDP4ZACU0U78/20160404/default/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=2e3c40e01716aa7d618644db6d033a96fc354257d26b864c46e1ee88b3f530a9', 'x-amz-date': '20160404T214839Z'}

rgw logs:

2016-04-04 14:48:39.381801 7f0bcffff700  1 ====== starting new request req=0x7f0bcfff97d0 =====
2016-04-04 14:48:39.381824 7f0bcffff700  2 req 4812:0.000024::GET /::initializing for trans_id = tx0000000000000000012cc-005702e137-1859-default
2016-04-04 14:48:39.381830 7f0bcffff700 10 host=pewpew.CENSORED
2016-04-04 14:48:39.381838 7f0bcffff700 20 subdomain=pewpew domain=CENSORED in_hosted_domain=1 in_hosted_domain_s3website=0
2016-04-04 14:48:39.381855 7f0bcffff700 10 meta>> HTTP_X_AMZ_CONTENT_SHA256
2016-04-04 14:48:39.381861 7f0bcffff700 10 meta>> HTTP_X_AMZ_DATE
2016-04-04 14:48:39.381867 7f0bcffff700 10 x>> x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2016-04-04 14:48:39.381870 7f0bcffff700 10 x>> x-amz-date:20160404T214839Z
2016-04-04 14:48:39.381896 7f0bcffff700 20 get_handler handler=25RGWHandler_REST_Bucket_S3
2016-04-04 14:48:39.381902 7f0bcffff700 10 handler=25RGWHandler_REST_Bucket_S3
2016-04-04 14:48:39.381904 7f0bcffff700  2 req 4812:0.000104:s3:GET /::getting op 0
2016-04-04 14:48:39.381912 7f0bcffff700 10 op=25RGWListBucket_ObjStore_S3
2016-04-04 14:48:39.381913 7f0bcffff700  2 req 4812:0.000113:s3:GET /:list_bucket:authorizing
2016-04-04 14:48:39.381924 7f0bcffff700 10 v4 signedheaders format = host;x-amz-content-sha256;x-amz-date
2016-04-04 14:48:39.381928 7f0bcffff700 10 v4 signature format = 2e3c40e01716aa7d618644db6d033a96fc354257d26b864c46e1ee88b3f530a9
2016-04-04 14:48:39.381936 7f0bcffff700 10 v4 credential format = JZXZ88D0YDP4ZACU0U78/20160404/default/s3/aws4_request
2016-04-04 14:48:39.381939 7f0bcffff700 10 access key id = JZXZ88D0YDP4ZACU0U78
2016-04-04 14:48:39.381941 7f0bcffff700 10 credential scope = 20160404/default/s3/aws4_request
2016-04-04 14:48:39.381988 7f0bcffff700 10 canonical headers format = host:pewpew.CENSORED
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20160404T214839Z

2016-04-04 14:48:39.382028 7f0bcffff700 10 payload request hash = e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2016-04-04 14:48:39.382051 7f0bcffff700 10 canonical request = GET
/pewpew/
delimiter=%2F
host:pewpew.CENSORED
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20160404T214839Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2016-04-04 14:48:39.382053 7f0bcffff700 10 canonical request hash = 9914877ef73e5ce2c41e571c9711aae9f329329e74174aeb3e163847abd24fb1
2016-04-04 14:48:39.382057 7f0bcffff700 10 string to sign = AWS4-HMAC-SHA256
20160404T214839Z
20160404/default/s3/aws4_request
9914877ef73e5ce2c41e571c9711aae9f329329e74174aeb3e163847abd24fb1
2016-04-04 14:48:39.382112 7f0bcffff700 10 date_k        = fe70e1eb03793f7ee8083a6c3c077b139fadca8d2acda7a2e90c443372dc1ba1
2016-04-04 14:48:39.382146 7f0bcffff700 10 region_k      = 058a25d53713b648eb0596463b353d7a6a8b25946809572a2e25b117688b67df
2016-04-04 14:48:39.382177 7f0bcffff700 10 service_k     = 954c1dcb71d4e11240de5da701cab84a2dd9d4704507da8a938d4f1c42ba0478
2016-04-04 14:48:39.382214 7f0bcffff700 10 signing_k     = 18fbccfcff43a4650cab1b6b6ed5322f4809e9e03e3147e2189d247d1a2c5964
2016-04-04 14:48:39.382250 7f0bcffff700 10 signature_k   = f6c8e7fa61af982556d96dd898d8bf9cb8a3695478ac381a8c0057cdf73d25a0
2016-04-04 14:48:39.382255 7f0bcffff700 10 new signature = f6c8e7fa61af982556d96dd898d8bf9cb8a3695478ac381a8c0057cdf73d25a0
2016-04-04 14:48:39.382256 7f0bcffff700 10 ----------------------------- Verifying signatures
2016-04-04 14:48:39.382257 7f0bcffff700 10 Signature     = 2e3c40e01716aa7d618644db6d033a96fc354257d26b864c46e1ee88b3f530a9
2016-04-04 14:48:39.382259 7f0bcffff700 10 New Signature = f6c8e7fa61af982556d96dd898d8bf9cb8a3695478ac381a8c0057cdf73d25a0
2016-04-04 14:48:39.382260 7f0bcffff700 10 -----------------------------
2016-04-04 14:48:39.382265 7f0bcffff700 10 failed to authorize request
2016-04-04 14:48:39.382267 7f0bcffff700 20 handler->ERRORHANDLER: err_no=-2027 new_err_no=-2027