Project

General

Profile

Bug #15358

rgw signature mismatch with escaped characters in url query portion

Added by Jonathan Fraser over 6 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
Immediate
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
rgw
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

using the aws-sdk-go to communicate with rados leads to some issues with escaped characters in the uri. Due to the nature of the sdk and it's use of absolute paths I have to use both the latest development branch as well as the apache frontend. When I send a request to the gateway with escaped characters in its query field I get a signature mismatch.

2016/04/01 16:40:30 DEBUG: Request s3/ListObjects Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET http://172.25.226.237:7481/pachbucket?prefix=pach%2F HTTP/1.1
Host: 172.25.226.237:7481
User-Agent: aws-sdk-go/1.1.12 (go1.6; linux; amd64)
Authorization: AWS4-HMAC-SHA256 Credential=2K1B762SFU1S2SX3L9IJ/20160401/default/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=38f31a3230b33b2ea6af8c8db321eec9b9d4afac09c2f8d7b9a805500f731e14
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20160401T234030Z
Accept-Encoding: gzip

-----------------------------------------------------
2016/04/01 16:40:30 DEBUG: Response s3/ListObjects Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 403 Forbidden
Content-Length: 196
Accept-Ranges: bytes
Content-Type: application/xml
Date: Fri, 01 Apr 2016 23:40:32 GMT
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5
X-Amz-Request-Id: tx00000000000000000004f-0056ff06f0-8267d-default

<Error><Code>SignatureDoesNotMatch</Code><RequestId>tx00000000000000000004f-0056ff06f0-8267d-default</RequestId><HostId>8267d-default-default</HostId></Error>
-----------------------------------------------------

without the escape it works fine:
2016/04/01 16:48:30 DEBUG: Request s3/ListObjects Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET http://172.25.226.237:7481/pachbucket?prefix=pach HTTP/1.1
Host: 172.25.226.237:7481
User-Agent: aws-sdk-go/1.1.12 (go1.6; linux; amd64)
Authorization: AWS4-HMAC-SHA256 Credential=2K1B762SFU1S2SX3L9IJ/20160401/default/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=8732568e19212239a5ed338b7062a8bf0a9244fcf874ddb1c1df5fc6e6f6fcb4
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20160401T234830Z
Accept-Encoding: gzip

-----------------------------------------------------
2016/04/01 16:48:30 DEBUG: Response s3/ListObjects Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Fri, 01 Apr 2016 23:48:32 GMT
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5
X-Amz-Request-Id: tx000000000000000000050-0056ff08d0-8267d-default

ef
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>pachbucket</Name><Prefix>pach</Prefix><Marker></Marker><MaxKeys>1000</MaxKeys><IsTruncated>false</IsTruncated></ListBucketResult>
0


test-bugfix-15358-aws.py View - Test bugfix against Amazon S3 (1.07 KB) Javier M. Mellid, 04/05/2016 10:45 PM

test-bugfix-15358-rgw.py View - Test bugfix against RGW (1.08 KB) Javier M. Mellid, 04/05/2016 10:45 PM

example-log-15358.txt View - Test prefix='logs%2F' case (3.53 KB) Javier M. Mellid, 04/05/2016 10:58 PM


Related issues

Related to rgw - Bug #15369: New AWSv4 signature support doesn't work: causes s3cmd to get 403/SignatureDoesNotMatch Resolved 04/04/2016

History

#1 Updated by Javier M. Mellid over 6 years ago

  • Assignee set to Javier M. Mellid

#2 Updated by Javier M. Mellid over 6 years ago

  • Status changed from New to Fix Under Review

Thanks Jonathan for the bug description. It was very useful. I was able to reproduce the bug with boto too.

A quick fix seems to resolve the issue. PR available at:

https://github.com/ceph/ceph/pull/8445

This PR could fix the bug http://tracker.ceph.com/issues/15369

#3 Updated by Sage Weil over 6 years ago

  • Priority changed from Normal to Immediate

#4 Updated by Nathan Cutler over 6 years ago

  • Related to Bug #15369: New AWSv4 signature support doesn't work: causes s3cmd to get 403/SignatureDoesNotMatch added

#5 Updated by Javier M. Mellid over 6 years ago

Sharing more testing on the bugfix.

It seems working as expected with my tests. With the bugfix in place, S3 and RGW show similar behaviour (see the example-log-15358.txt attachment) when running the scripts/tests attached. I used prefix='logs%2F' to raise the bug.

Both test cases (written in Python) should be useful to include as part of s3-tests if needed (the original report uses aws-sdk-go though)

#6 Updated by Orit Wasserman over 6 years ago

  • Status changed from Fix Under Review to Resolved

Also available in: Atom PDF