Project

General

Profile

Actions
Copy link

Bug #44804

closed

Signed Url generated with metadata alway return SignatureNotMatch

Added by hoan nv about 4 years ago. Updated about 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Or Friedmann
Target version:
Ceph - v13.2.7
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
Ceph - v13.2.7, Ceph - v14.2.7
ceph-qa-suite:
rgw
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Hi all.

1. Upload without metadata (successfully)
I use my code to generate url

s3client = boto3.client(service_name='s3',
                  aws_access_key_id=access_key,
                  aws_secret_access_key=secret_key,
                  endpoint_url=endpoint_url, use_ssl=False, verify=False,
                  config=Config(signature_version='s3v4'))
bucket_name = 'bucket3'
s3client.create_bucket(Bucket=bucket_name)
url = s3client.generate_presigned_url('put_object', Params={'Bucket':bucket_name, 'Key':'1.jpg'}, ExpiresIn=3600, HttpMethod='PUT')
print(url)

I have presigned url to put object

http://10.5.8.156:80/bucket3/1.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Credential=74AWMS8A1AP0MZYKQO0W%2F20200330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-SignedHeaders=host&X-Amz-Date=20200330T073700Z&X-Amz-Signature=b3727e6826c14fa4fca3ba10af3185eef8d891ecc865586831965a25b626997b

Then i use curl 

curl --request PUT --upload-file 1.jpg "http://10.5.8.156:80/bucket3/1.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Credential=74AWMS8A1AP0MZYKQO0W%2F20200330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-SignedHeaders=host&X-Amz-Date=20200330T073700Z&X-Amz-Signature=b3727e6826c14fa4fca3ba10af3185eef8d891ecc865586831965a25b626997b"

I upload file successfully.

2. Upload with metadata (403 SignatureNotMatch)

But I add more header metadata

url = s3client.generate_presigned_url('put_object', Params={'Bucket':bucket_name, 'Key':'1.jpg','ContentType':'image/jpeg','ACL':'public-read'}, ExpiresIn=3600, HttpMethod='PUT')
print(url)

Request return 403 SignatureNotMatch

This is log debug boto3

2020-03-30 14:44:05,027 botocore.auth [DEBUG] Calculating signature using v4 auth.
2020-03-30 14:44:05,028 botocore.auth [DEBUG] CanonicalRequest:
PUT
/bucket3/1.jpg
X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=74AWMS8A1AP0MZYKQO0W%2F20200330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200330T074405Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl
content-type:image/jpeg
host:10.5.8.156
x-amz-acl:public-read

content-type;host;x-amz-acl
UNSIGNED-PAYLOAD
2020-03-30 14:44:05,028 botocore.auth [DEBUG] StringToSign:
AWS4-HMAC-SHA256
20200330T074405Z
20200330/us-east-1/s3/aws4_request
c8ee46724d869e942869f20b320b83b4e2ea17dca25055d75805ff891ca72391
2020-03-30 14:44:05,029 botocore.auth [DEBUG] Signature:
4052d1f932ead5d54133a50880fb8e8b3420b3f9e013873bb4bf33c744a8b09f
http://10.5.8.156:80/bucket3/1.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Credential=74AWMS8A1AP0MZYKQO0W%2F20200330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl&X-Amz-Date=20200330T074405Z&X-Amz-Signature=4052d1f932ead5d54133a50880fb8e8b3420b3f9e013873bb4bf33c744a8b09f

Then i use curl command to upload

curl --request PUT --upload-file 1.jpg "http://10.5.8.156:80/bucket3/1.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Credential=74AWMS8A1AP0MZYKQO0W%2F20200330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl&X-Amz-Date=20200330T074405Z&X-Amz-Signature=4052d1f932ead5d54133a50880fb8e8b3420b3f9e013873bb4bf33c744a8b09f" 

It rerturn :
<?xml version="1.0" encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch</Code><RequestId>tx000000000000000000038-005e81a380-2122-default</RequestId><HostId>2122-default-default</HostId>

This is rgw debug log.

2020-03-30 14:45:04.323 7f4bdbcf0700 10 canonical headers format = host:10.5.8.156

2020-03-30 14:45:04.323 7f4bdbcf0700 10 payload request hash = UNSIGNED-PAYLOAD
2020-03-30 14:45:04.323 7f4bdbcf0700 10 canonical request = PUT
/bucket3/1.jpg
X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=74AWMS8A1AP0MZYKQO0W%2F20200330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200330T074405Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl
host:10.5.8.156

content-type;host;x-amz-acl
UNSIGNED-PAYLOAD
2020-03-30 14:45:04.323 7f4bdbcf0700 10 canonical request hash = 15bf4c3b375c85b3a785a62ed1961f545f99b0a73571118953196b4272f12b22
2020-03-30 14:45:04.323 7f4bdbcf0700 10 string to sign = AWS4-HMAC-SHA256
20200330T074405Z
20200330/us-east-1/s3/aws4_request
15bf4c3b375c85b3a785a62ed1961f545f99b0a73571118953196b4272f12b22
2020-03-30 14:45:04.323 7f4bdbcf0700 10 date_k    = b24064027f23cd98c3255b42b2ee511d907663f0e29c42e15e8f2b843a231cbc
2020-03-30 14:45:04.323 7f4bdbcf0700 10 region_k  = 4a1865641ecc0ab67a6dced8b6d09822752c873b8a5d19bacb6139f10a01e7dd
2020-03-30 14:45:04.323 7f4bdbcf0700 10 service_k = a2eff4dc70d9f170de38439e4732287358c3ebd821d9d354c188ca409a09fbbb
2020-03-30 14:45:04.323 7f4bdbcf0700 10 signing_k = 6be6df1acb55821c2571af97a21845caf277ce8007103d7bde1188c5b0403528
2020-03-30 14:45:04.323 7f4bdbcf0700 10 generated signature = 6ad1438a1eded893f0f89153efeaf00d97129d99bd4c2f9467096704f8cd7e57
2020-03-30 14:45:04.323 7f4bdbcf0700  5 Failed the auth strategy, reason=-2027
2020-03-30 14:45:04.323 7f4bdbcf0700 10 failed to authorize request
2020-03-30 14:45:04.327 7f4bdbcf0700  2 req 56:0.00399995s:s3:PUT /bucket3/1.jpg:put_obj:op status=0
2020-03-30 14:45:04.327 7f4bdbcf0700  2 req 56:0.00399995s:s3:PUT /bucket3/1.jpg:put_obj:http status=403

3. Upload with metadata successfully (add more header in curl command to work arround)

curl --request PUT --upload-file 1.jpg  -H 'Content-Type:image/jpeg' -H 'x-amz-acl:public-read' "http://10.5.8.156:80/bucket3/1.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Credential=74AWMS8A1AP0MZYKQO0W%2F20200330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl&X-Amz-Date=20200330T074405Z&X-Amz-Signature=4052d1f932ead5d54133a50880fb8e8b3420b3f9e013873bb4bf33c744a8b09f"

This is rgw log

2020-03-30 14:50:09.071 7f4bdbcf0700 10 credential scope = 20200330/us-east-1/s3/aws4_request
2020-03-30 14:50:09.071 7f4bdbcf0700 10 canonical headers format = content-type:image/jpeg
host:10.5.8.156
x-amz-acl:public-read

2020-03-30 14:50:09.071 7f4bdbcf0700 10 payload request hash = UNSIGNED-PAYLOAD
2020-03-30 14:50:09.075 7f4bdbcf0700 10 canonical request = PUT
/bucket3/1.jpg
X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=74AWMS8A1AP0MZYKQO0W%2F20200330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200330T074405Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl
content-type:image/jpeg
host:10.5.8.156
x-amz-acl:public-read

content-type;host;x-amz-acl
UNSIGNED-PAYLOAD
2020-03-30 14:50:09.075 7f4bdbcf0700 10 canonical request hash = c8ee46724d869e942869f20b320b83b4e2ea17dca25055d75805ff891ca72391
2020-03-30 14:50:09.075 7f4bdbcf0700 10 string to sign = AWS4-HMAC-SHA256
20200330T074405Z
20200330/us-east-1/s3/aws4_request
c8ee46724d869e942869f20b320b83b4e2ea17dca25055d75805ff891ca72391

Rgw not add metadata from url in 

http://10.5.8.156:80/bucket3/1.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Credential=74AWMS8A1AP0MZYKQO0W%2F20200330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl&X-Amz-Date=20200330T074405Z&X-Amz-Signature=4052d1f932ead5d54133a50880fb8e8b3420b3f9e013873bb4bf33c744a8b09flink

to canonical request.
I think rgw need add parameter from query string in signed url to canonical request.

Thanks.

Actions
Copy link

Also available in: Atom PDF