Project

General

Profile

Actions
Copy link

Bug #15358

closed

rgw signature mismatch with escaped characters in url query portion

Added by Jonathan Fraser about 8 years ago. Updated about 8 years ago.

Status:
Resolved
Priority:
Immediate
Assignee:
Javier M. Mellid
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
Ceph - v10.1.1
ceph-qa-suite:
rgw
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

using the aws-sdk-go to communicate with rados leads to some issues with escaped characters in the uri. Due to the nature of the sdk and it's use of absolute paths I have to use both the latest development branch as well as the apache frontend. When I send a request to the gateway with escaped characters in its query field I get a signature mismatch.

2016/04/01 16:40:30 DEBUG: Request s3/ListObjects Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET http://172.25.226.237:7481/pachbucket?prefix=pach%2F HTTP/1.1
Host: 172.25.226.237:7481
User-Agent: aws-sdk-go/1.1.12 (go1.6; linux; amd64)
Authorization: AWS4-HMAC-SHA256 Credential=2K1B762SFU1S2SX3L9IJ/20160401/default/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=38f31a3230b33b2ea6af8c8db321eec9b9d4afac09c2f8d7b9a805500f731e14
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20160401T234030Z
Accept-Encoding: gzip

-----------------------------------------------------
2016/04/01 16:40:30 DEBUG: Response s3/ListObjects Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 403 Forbidden
Content-Length: 196
Accept-Ranges: bytes
Content-Type: application/xml
Date: Fri, 01 Apr 2016 23:40:32 GMT
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5
X-Amz-Request-Id: tx00000000000000000004f-0056ff06f0-8267d-default

<Error><Code>SignatureDoesNotMatch</Code><RequestId>tx00000000000000000004f-0056ff06f0-8267d-default</RequestId><HostId>8267d-default-default</HostId></Error>
-----------------------------------------------------

without the escape it works fine:
2016/04/01 16:48:30 DEBUG: Request s3/ListObjects Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET http://172.25.226.237:7481/pachbucket?prefix=pach HTTP/1.1
Host: 172.25.226.237:7481
User-Agent: aws-sdk-go/1.1.12 (go1.6; linux; amd64)
Authorization: AWS4-HMAC-SHA256 Credential=2K1B762SFU1S2SX3L9IJ/20160401/default/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=8732568e19212239a5ed338b7062a8bf0a9244fcf874ddb1c1df5fc6e6f6fcb4
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20160401T234830Z
Accept-Encoding: gzip

-----------------------------------------------------
2016/04/01 16:48:30 DEBUG: Response s3/ListObjects Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Fri, 01 Apr 2016 23:48:32 GMT
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5
X-Amz-Request-Id: tx000000000000000000050-0056ff08d0-8267d-default

ef
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/&quot;&gt;&lt;Name&gt;pachbucket&lt;/Name&gt;&lt;Prefix&gt;pach&lt;/Prefix&gt;&lt;Marker&gt;&lt;/Marker&gt;&lt;MaxKeys&gt;1000&lt;/MaxKeys&gt;&lt;IsTruncated&gt;false&lt;/IsTruncated&gt;&lt;/ListBucketResult>
0

Files

Download all files
test-bugfix-15358-aws.py (1.07 KB) test-bugfix-15358-aws.py Test bugfix against Amazon S3 Javier M. Mellid, 04/05/2016 10:45 PM
test-bugfix-15358-rgw.py (1.08 KB) test-bugfix-15358-rgw.py Test bugfix against RGW Javier M. Mellid, 04/05/2016 10:45 PM
example-log-15358.txt (3.53 KB) example-log-15358.txt Test prefix='logs%2F' case Javier M. Mellid, 04/05/2016 10:58 PM

Related issues 1 (0 open — 1 closed)

Related to rgw - Bug #15369: New AWSv4 signature support doesn't work: causes s3cmd to get 403/SignatureDoesNotMatchResolvedJavier M. Mellid04/04/2016

Actions
Actions
Copy link
 #1

Updated by Javier M. Mellid about 8 years ago

  • Assignee set to Javier M. Mellid
Actions
Copy link
 #2

Updated by Javier M. Mellid about 8 years ago

  • Status changed from New to Fix Under Review

Thanks Jonathan for the bug description. It was very useful. I was able to reproduce the bug with boto too.

A quick fix seems to resolve the issue. PR available at:

https://github.com/ceph/ceph/pull/8445

This PR could fix the bug http://tracker.ceph.com/issues/15369

Actions
Copy link
 #3

Updated by Sage Weil about 8 years ago

  • Priority changed from Normal to Immediate
Actions
Copy link
 #4

Updated by Nathan Cutler about 8 years ago

  • Related to Bug #15369: New AWSv4 signature support doesn't work: causes s3cmd to get 403/SignatureDoesNotMatch added
Actions
Copy link Download all files
 #5

Updated by Javier M. Mellid about 8 years ago

Sharing more testing on the bugfix.

It seems working as expected with my tests. With the bugfix in place, S3 and RGW show similar behaviour (see the example-log-15358.txt attachment) when running the scripts/tests attached. I used prefix='logs%2F' to raise the bug.

Both test cases (written in Python) should be useful to include as part of s3-tests if needed (the original report uses aws-sdk-go though)

Actions
Copy link
 #6

Updated by Orit Wasserman about 8 years ago

  • Status changed from Fix Under Review to Resolved
Actions
Copy link

Also available in: Atom PDF