Project

General

Profile

Feature #15975

RGW bucket name logic should fall back to HTTP hostname

Added by Robin Johnson over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
High
Assignee:
Target version:
-
Start date:
05/20/2016
Due date:
% Done:

0%

Source:
other
Tags:
staticsites
Backport:
jewel
Reviewed:
User Impact:
Affected Versions:
Release:
jewel, master
Needs Doc:
No

Description

The VHost-style bucket logic presently requires that there is an accessible CNAME entry, that has the name of a bucket on the right hand side, eg:

www.bucket.com. IN CNAME www.bucket.com.s3.storage.com.
demo.bucket.com. IN CNAME some-other-bucket-name.s3.storage.com.

It does NOT work for the following cases:
1.
DNS entries directly to the storage.

www.bucket.com. IN CNAME s3.storage.com.

2.
Root entries of domains, where CNAME cannot be used. This is common in static website hosting.

bucket.com IN SOA ( ... )
bucket.com IN NS ( ... )
bucket.com IN A 1.2.3.4 # Same IP as s3.storage.com

3.
Private DNS where from the address is resolvable from the perspective of the client, but NOT the RGW instance.

For these cases, the bucket name logic SHOULD fall back to trying to use value of the HTTP Host header as the bucket name, with no modification.


Related issues

Copied to rgw - Backport #16182: jewel: backport static sites fixes master->jewel Resolved

History

#1 Updated by Robin Johnson over 1 year ago

The logic in preprocess is very messy, and ripe for rewrite to reflect this improvement.
1. Is $Host an exact match to the suffix lists -> path-style access; bucket name is the first path element (Client URL http://$suffix/$bucket/...)
2. Is $Host a suffix-match against lists -> subdomain access; Bucket name is $Host ~= s/${suffix}$//; (Client URL http://$bucket.$suffix/...)
3. Resolve $Host, store result as $CNameHost
4. Is $CNameHost an exact match to the suffix lists -> Vhost-style access; bucket name is $Host (Client URL http://$bucket/...)
5. Is $CNameHost a suffix match against lists -> subdomain access; bucket name is CNameHost ~= s/${suffix}$// (Client URL http://$bucket/...)
6. Fallback to use $Host as bucket (Client URL http://$bucket/...)

#2 Updated by Robin Johnson over 1 year ago

Phase 1 implementation (Kraken, backport to Jewel):
Add fallback to $Host if subdomain == NULL/empty at the end of the present logic.

Phase 2 implementation (Kraken):
Rewrite decision logic to support all the above cases w/ staticsites clearly.

#3 Updated by Robin Johnson over 1 year ago

I reviewed AWS S3, both for documentation and actual functionality (since they can differ).

This page covers what we're talking about.
http://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html#VirtualHostingCustomURLs

The CNAME DNS record should alias your domain name to the appropriate virtual hosted–style host name. For example, if your bucket name and domain name are images.johnsmith.net, the CNAME record should alias to images.johnsmith.net.s3.amazonaws.com.

www.example.com. IN CNAME www.example.com.s3.amazonaws.com.
Works in AWS-S3.
Works in Ceph-Hammer.

Based on this, we could consider disabling the rgw_resolve_cname functionality for stronger S3 functionality matching.
Works in Ceph-Jewel.

Setting the alias target to s3.amazonaws.com also works, but it may result in extra HTTP redirects

Confirmed:
www.example.com. IN CNAME s3.amazonaws.com.
Works in AWS-S3.
Does NOT work in Ceph-Hammer.
Does NOT work in Ceph-Jewel.

Because Amazon S3 sees only the original host name www.example.com and is unaware of the CNAME mapping used to resolve the request, the CNAME and the bucket name must be the same.
...
The bucket name must exactly match the host name.

This deviates from something that RGW can do:
www.example.com. IN CNAME other-bucket-name.s3.rgw.ceph.com.
Works in Ceph.
Does NOT work in S3.

#4 Updated by Robin Johnson over 1 year ago

Phase1 implementation available in https://github.com/ceph/ceph/pull/9354

It's critical for staticsites, but also fixes a case where AWS S3 worked and RGW did not.

#5 Updated by Nathan Cutler over 1 year ago

  • Copied from Backport #16182: jewel: backport static sites fixes master->jewel added

#6 Updated by Nathan Cutler over 1 year ago

  • Copied from deleted (Backport #16182: jewel: backport static sites fixes master->jewel)

#7 Updated by Nathan Cutler over 1 year ago

  • Copied to Backport #16182: jewel: backport static sites fixes master->jewel added

#8 Updated by Loic Dachary over 1 year ago

  • Status changed from New to In Progress

#9 Updated by Loic Dachary over 1 year ago

  • Status changed from In Progress to Resolved

Also available in: Atom PDF