Project

General

Profile

Feature #10333

rgw: support AWS4 authentication

Added by Yehuda Sadeh almost 9 years ago. Updated over 7 years ago.

Status:
Resolved
Priority:
Normal
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
Reviewed:
Affected Versions:
Pull request ID:

Related issues

Related to rgw - Bug #10195: s3 java jdk conn.getobject(...) (get s3 object) method fails with latest version of aws s3 java jdk Closed 11/26/2014
Duplicated by rgw - Feature #11858: Support Amazon v4 Signatures Duplicate 06/03/2015

Associated revisions

Revision 9a0de484 (diff)
Added by Javier M. Mellid almost 8 years ago

rgw: AWS4 authentication minimal support

Amazon S3 supports Signature Version 4. This patch contains the minimal
implementation supporting AWS4 in RGW. It implements AWS4 authentication
on http methods without body content and vars not shipping in the
request query string.

Fixes: #10333

Signed-off-by: Javier M. Mellid <>

Revision cbfc4af6 (diff)
Added by Javier M. Mellid almost 8 years ago

rgw: handle AWS4 auth case when query string exists

Crafts the canonical query string. URI-encode each parameter name and
value properly.

Fixes: #10333

Signed-off-by: Javier M. Mellid <>

Revision a15abe46 (diff)
Added by Javier M. Mellid almost 8 years ago

rgw: add proper AWSv4 and AWSv2 auth detection

Adds a new authorize function to identify/handle AWSv4 and AWSv2 auth
properly, handling common code, etc.

Fixes: #10333

Signed-off-by: Javier M. Mellid <>

Revision 49856eb7 (diff)
Added by Javier M. Mellid almost 8 years ago

rgw: AWS4 auth support when using request params

Fixes: #10333

Signed-off-by: Javier M. Mellid <>

Revision 483ad815 (diff)
Added by Javier M. Mellid almost 8 years ago

rgw: avoid re-encoding already encoded query strings in AWS4 auth

When computing V4 signature, we need to encode the query string. But it
could come already encoded, at least partially.

So do not encode the entities that are already encoded.

Fixes: #10333

Signed-off-by: Javier M. Mellid <>

Revision 466cb81c (diff)
Added by Javier M. Mellid almost 8 years ago

rgw: UNSIGNED-PAYLOAD support in AWS4 auth

Fixes: #10333

Signed-off-by: Javier M. Mellid <>

Revision 3210cd8a (diff)
Added by Javier M. Mellid almost 8 years ago

rgw: AWS4 auth support for positive content-length

Handle AWS4 auth on requests with positive content-length. It splits the
auth process along several steps to process the body content on the fly
instead of reading the whole body in memory. After that, it completes
the delayed AWS4 auth properly.

Requests with content-length <= 0 are validated as usual. They don't
require any kind of completion.

Requests with content-length > 0 use a streaming approach together with
a completion step.

Fixes: #10333

Signed-off-by: Javier M. Mellid <>

Revision b7772cac (diff)
Added by Javier M. Mellid almost 8 years ago

rgw: initialize aws4_auth_complete

Fixes: #10333

Signed-off-by: Javier M. Mellid <>

Revision c32e5eaa (diff)
Added by Javier M. Mellid almost 8 years ago

rgw: verify Content-MD5 is a valid base64 value

Fixes: #10333

Signed-off-by: Javier M. Mellid <>

Revision 79b6b4f1 (diff)
Added by Javier M. Mellid almost 8 years ago

rgw: check if x-amz-date has a value

Fixes: #10333

Signed-off-by: Javier M. Mellid <>

Revision 2de292ac (diff)
Added by Javier M. Mellid almost 8 years ago

rgw: validate x-amz-sha256

Verify content's sha256 sum matches the expected value.

Fixes: #10333

Signed-off-by: Javier M. Mellid <>

Revision 4da85395 (diff)
Added by Javier M. Mellid almost 8 years ago

rgw: achieve same error behaviour in S3 and RGW when processing signedheaders

This patch gets the same error response in S3 and RGW when the error is related
to the signedheaders processing.

Fixes: #10333

Signed-off-by: Javier M. Mellid <>

History

#1 Updated by Yehuda Sadeh almost 9 years ago

  • Subject changed from rgw: support AWS4 to rgw: support AWS4 authentication

#2 Updated by Javier M. Mellid over 8 years ago

I started this feature and I have some initial bits working now but I don't have the right permissions to ask for assignation or update the "Assignee" field in tracker.

Would it be possible taking care of this feature implementation? If so, anybody updating the 'Assignee' field to jmunhoz? Thanks!

#3 Updated by Yehuda Sadeh over 8 years ago

  • Assignee set to Javier M. Mellid

Done. You should be able to modify these fields under the rgw project now.

#4 Updated by Javier M. Mellid over 8 years ago

Thanks Yehuda.

#5 Updated by Javier M. Mellid over 8 years ago

I've just created a pull request to merge a first version for supporting Amazon's S3 signature V4 (from now on, AWS4).

The patch implements the basic required primitives to work with AWS4 (sha256 crypto, encoding, and so on), and is able to authenticate HTTP requests that don't include the parameters in the query string (only as HTTP headers) and that do not contain a body. This is work in progress, and I hope to submit a new pull request for this part as soon as possible.

The patch is fully mergeable, so if done it will partially support V4. As said before, I am working on bringing the remaining features.

Note that V2 authentication continues to work as usual.

Any feedback is welcomed.

Pull request is available at:

https://github.com/ceph/ceph/pull/4943

#6 Updated by Josh Durgin over 8 years ago

  • Status changed from New to Fix Under Review

#7 Updated by Javier M. Mellid over 8 years ago

Pull request updated with the missing bits. It contains the whole feature.

#8 Updated by Sage Weil over 7 years ago

  • Status changed from Fix Under Review to Resolved

Also available in: Atom PDF