Feature #15975: RGW bucket name logic should fall back to HTTP hostname - rgw - Ceph

Actions

Copy link

Feature #15975

closed

RGW bucket name logic should fall back to HTTP hostname

Added by Robin Johnson almost 8 years ago. Updated over 7 years ago.

Status:

Resolved

Priority:

High

Assignee:

Robin Johnson

Target version:

% Done:

Source:

other

Tags:

staticsites

Backport:

jewel

Reviewed:

Affected Versions:

Pull request ID:

Description

The VHost-style bucket logic presently requires that there is an accessible CNAME entry, that has the name of a bucket on the right hand side, eg:

www.bucket.com. IN CNAME www.bucket.com.s3.storage.com.
demo.bucket.com. IN CNAME some-other-bucket-name.s3.storage.com.

It does NOT work for the following cases:
1.
DNS entries directly to the storage.

www.bucket.com. IN CNAME s3.storage.com.

2.
Root entries of domains, where CNAME cannot be used. This is common in static website hosting.

bucket.com IN SOA ( ... )
bucket.com IN NS ( ... )
bucket.com IN A 1.2.3.4 # Same IP as s3.storage.com

3.
Private DNS where from the address is resolvable from the perspective of the client, but NOT the RGW instance.

For these cases, the bucket name logic SHOULD fall back to trying to use value of the HTTP Host header as the bucket name, with no modification.

Related issues 1 (0 open — 1 closed)

Actions

Copy link

Updated by Robin Johnson almost 8 years ago

The logic in preprocess is very messy, and ripe for rewrite to reflect this improvement.
1. Is $Host an exact match to the suffix lists -> path-style access; bucket name is the first path element (Client URL http://$suffix/$bucket/...)
2. Is $Host a suffix-match against lists -> subdomain access; Bucket name is $Host ~= s/${suffix}$//; (Client URL http://$bucket.$suffix/...)
3. Resolve $Host, store result as $CNameHost
4. Is $CNameHost an exact match to the suffix lists -> Vhost-style access; bucket name is $Host (Client URL http://$bucket/...)
5. Is $CNameHost a suffix match against lists -> subdomain access; bucket name is CNameHost ~= s/${suffix}$// (Client URL http://$bucket/...)
6. Fallback to use $Host as bucket (Client URL http://$bucket/...)

Actions

Copy link

Updated by Robin Johnson almost 8 years ago

Phase 1 implementation (Kraken, backport to Jewel):
Add fallback to $Host if subdomain == NULL/empty at the end of the present logic.

Phase 2 implementation (Kraken):
Rewrite decision logic to support all the above cases w/ staticsites clearly.

Actions

Copy link

Updated by Robin Johnson almost 8 years ago

I reviewed AWS S3, both for documentation and actual functionality (since they can differ).

This page covers what we're talking about.
http://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html#VirtualHostingCustomURLs

The CNAME DNS record should alias your domain name to the appropriate virtual hosted–style host name. For example, if your bucket name and domain name are images.johnsmith.net, the CNAME record should alias to images.johnsmith.net.s3.amazonaws.com.

www.example.com. IN CNAME www.example.com.s3.amazonaws.com.
Works in AWS-S3.
Works in Ceph-Hammer.

Based on this, we could consider disabling the rgw_resolve_cname functionality for stronger S3 functionality matching.
Works in Ceph-Jewel.

Setting the alias target to s3.amazonaws.com also works, but it may result in extra HTTP redirects

Confirmed:
www.example.com. IN CNAME s3.amazonaws.com.
Works in AWS-S3.
Does NOT work in Ceph-Hammer.
Does NOT work in Ceph-Jewel.

Because Amazon S3 sees only the original host name www.example.com and is unaware of the CNAME mapping used to resolve the request, the CNAME and the bucket name must be the same.
...
The bucket name must exactly match the host name.

This deviates from something that RGW can do:
www.example.com. IN CNAME other-bucket-name.s3.rgw.ceph.com.
Works in Ceph.
Does NOT work in S3.

Actions

Copy link