Actions
Bug #44804
closedSigned Url generated with metadata alway return SignatureNotMatch
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
rgw
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Hi all.
1. Upload without metadata (successfully)
I use my code to generate url
s3client = boto3.client(service_name='s3', aws_access_key_id=access_key, aws_secret_access_key=secret_key, endpoint_url=endpoint_url, use_ssl=False, verify=False, config=Config(signature_version='s3v4')) bucket_name = 'bucket3' s3client.create_bucket(Bucket=bucket_name) url = s3client.generate_presigned_url('put_object', Params={'Bucket':bucket_name, 'Key':'1.jpg'}, ExpiresIn=3600, HttpMethod='PUT') print(url)
I have presigned url to put object
http://10.5.8.156:80/bucket3/1.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Credential=74AWMS8A1AP0MZYKQO0W%2F20200330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-SignedHeaders=host&X-Amz-Date=20200330T073700Z&X-Amz-Signature=b3727e6826c14fa4fca3ba10af3185eef8d891ecc865586831965a25b626997b
Then i use curl
curl --request PUT --upload-file 1.jpg "http://10.5.8.156:80/bucket3/1.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Credential=74AWMS8A1AP0MZYKQO0W%2F20200330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-SignedHeaders=host&X-Amz-Date=20200330T073700Z&X-Amz-Signature=b3727e6826c14fa4fca3ba10af3185eef8d891ecc865586831965a25b626997b"
I upload file successfully.
2. Upload with metadata (403 SignatureNotMatch)
But I add more header metadata
url = s3client.generate_presigned_url('put_object', Params={'Bucket':bucket_name, 'Key':'1.jpg','ContentType':'image/jpeg','ACL':'public-read'}, ExpiresIn=3600, HttpMethod='PUT') print(url)
Request return 403 SignatureNotMatch
This is log debug boto3
2020-03-30 14:44:05,027 botocore.auth [DEBUG] Calculating signature using v4 auth. 2020-03-30 14:44:05,028 botocore.auth [DEBUG] CanonicalRequest: PUT /bucket3/1.jpg X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=74AWMS8A1AP0MZYKQO0W%2F20200330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200330T074405Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl content-type:image/jpeg host:10.5.8.156 x-amz-acl:public-read content-type;host;x-amz-acl UNSIGNED-PAYLOAD 2020-03-30 14:44:05,028 botocore.auth [DEBUG] StringToSign: AWS4-HMAC-SHA256 20200330T074405Z 20200330/us-east-1/s3/aws4_request c8ee46724d869e942869f20b320b83b4e2ea17dca25055d75805ff891ca72391 2020-03-30 14:44:05,029 botocore.auth [DEBUG] Signature: 4052d1f932ead5d54133a50880fb8e8b3420b3f9e013873bb4bf33c744a8b09f http://10.5.8.156:80/bucket3/1.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Credential=74AWMS8A1AP0MZYKQO0W%2F20200330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl&X-Amz-Date=20200330T074405Z&X-Amz-Signature=4052d1f932ead5d54133a50880fb8e8b3420b3f9e013873bb4bf33c744a8b09f
Then i use curl command to upload
curl --request PUT --upload-file 1.jpg "http://10.5.8.156:80/bucket3/1.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Credential=74AWMS8A1AP0MZYKQO0W%2F20200330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl&X-Amz-Date=20200330T074405Z&X-Amz-Signature=4052d1f932ead5d54133a50880fb8e8b3420b3f9e013873bb4bf33c744a8b09f" It rerturn : <?xml version="1.0" encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch</Code><RequestId>tx000000000000000000038-005e81a380-2122-default</RequestId><HostId>2122-default-default</HostId>
This is rgw debug log.
2020-03-30 14:45:04.323 7f4bdbcf0700 10 canonical headers format = host:10.5.8.156 2020-03-30 14:45:04.323 7f4bdbcf0700 10 payload request hash = UNSIGNED-PAYLOAD 2020-03-30 14:45:04.323 7f4bdbcf0700 10 canonical request = PUT /bucket3/1.jpg X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=74AWMS8A1AP0MZYKQO0W%2F20200330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200330T074405Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl host:10.5.8.156 content-type;host;x-amz-acl UNSIGNED-PAYLOAD 2020-03-30 14:45:04.323 7f4bdbcf0700 10 canonical request hash = 15bf4c3b375c85b3a785a62ed1961f545f99b0a73571118953196b4272f12b22 2020-03-30 14:45:04.323 7f4bdbcf0700 10 string to sign = AWS4-HMAC-SHA256 20200330T074405Z 20200330/us-east-1/s3/aws4_request 15bf4c3b375c85b3a785a62ed1961f545f99b0a73571118953196b4272f12b22 2020-03-30 14:45:04.323 7f4bdbcf0700 10 date_k = b24064027f23cd98c3255b42b2ee511d907663f0e29c42e15e8f2b843a231cbc 2020-03-30 14:45:04.323 7f4bdbcf0700 10 region_k = 4a1865641ecc0ab67a6dced8b6d09822752c873b8a5d19bacb6139f10a01e7dd 2020-03-30 14:45:04.323 7f4bdbcf0700 10 service_k = a2eff4dc70d9f170de38439e4732287358c3ebd821d9d354c188ca409a09fbbb 2020-03-30 14:45:04.323 7f4bdbcf0700 10 signing_k = 6be6df1acb55821c2571af97a21845caf277ce8007103d7bde1188c5b0403528 2020-03-30 14:45:04.323 7f4bdbcf0700 10 generated signature = 6ad1438a1eded893f0f89153efeaf00d97129d99bd4c2f9467096704f8cd7e57 2020-03-30 14:45:04.323 7f4bdbcf0700 5 Failed the auth strategy, reason=-2027 2020-03-30 14:45:04.323 7f4bdbcf0700 10 failed to authorize request 2020-03-30 14:45:04.327 7f4bdbcf0700 2 req 56:0.00399995s:s3:PUT /bucket3/1.jpg:put_obj:op status=0 2020-03-30 14:45:04.327 7f4bdbcf0700 2 req 56:0.00399995s:s3:PUT /bucket3/1.jpg:put_obj:http status=403
3. Upload with metadata successfully (add more header in curl command to work arround)
curl --request PUT --upload-file 1.jpg -H 'Content-Type:image/jpeg' -H 'x-amz-acl:public-read' "http://10.5.8.156:80/bucket3/1.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Credential=74AWMS8A1AP0MZYKQO0W%2F20200330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl&X-Amz-Date=20200330T074405Z&X-Amz-Signature=4052d1f932ead5d54133a50880fb8e8b3420b3f9e013873bb4bf33c744a8b09f"
This is rgw log
2020-03-30 14:50:09.071 7f4bdbcf0700 10 credential scope = 20200330/us-east-1/s3/aws4_request 2020-03-30 14:50:09.071 7f4bdbcf0700 10 canonical headers format = content-type:image/jpeg host:10.5.8.156 x-amz-acl:public-read 2020-03-30 14:50:09.071 7f4bdbcf0700 10 payload request hash = UNSIGNED-PAYLOAD 2020-03-30 14:50:09.075 7f4bdbcf0700 10 canonical request = PUT /bucket3/1.jpg X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=74AWMS8A1AP0MZYKQO0W%2F20200330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200330T074405Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl content-type:image/jpeg host:10.5.8.156 x-amz-acl:public-read content-type;host;x-amz-acl UNSIGNED-PAYLOAD 2020-03-30 14:50:09.075 7f4bdbcf0700 10 canonical request hash = c8ee46724d869e942869f20b320b83b4e2ea17dca25055d75805ff891ca72391 2020-03-30 14:50:09.075 7f4bdbcf0700 10 string to sign = AWS4-HMAC-SHA256 20200330T074405Z 20200330/us-east-1/s3/aws4_request c8ee46724d869e942869f20b320b83b4e2ea17dca25055d75805ff891ca72391
Rgw not add metadata from url in
http://10.5.8.156:80/bucket3/1.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Credential=74AWMS8A1AP0MZYKQO0W%2F20200330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl&X-Amz-Date=20200330T074405Z&X-Amz-Signature=4052d1f932ead5d54133a50880fb8e8b3420b3f9e013873bb4bf33c744a8b09flink
to canonical request.
I think rgw need add parameter from query string in signed url to canonical request.
Thanks.
Actions