PowerDNS backend for RGW¶
Summary¶
When using multiple RGW regions it's hard to direct traffic to the correct region. If a region receives a request which is for a different region it will answer with a 301 HTTP redirect, but this adds extra TCP and HTTP traffic which increases latency.
Wrong configuration on the users end could also lead to additional HTTP traffic on a region since it's only responding with 301s instead of serving objects.
Owners¶
- Wido den Hollander (Owner)
- Name (Affiliation)
- Name
Interested Parties¶
- Name (Affiliation)
- Name (Affiliation)
- Name
Current Status¶
Only an idea, but could be turned into code rather quickly.
Will start with a PoC which won't perform rather well, after that a better version can be written.
Update: POC written and available as Gist on Github: https://gist.github.com/wido/7664859
Detailed Description¶
A backend for PowerDNS could solve this where the DNS server answers with a different CNAME depending in the bucket in the hostname.
The backend has to be aware of the regions and buckets to be able to direct traffic.
For example, a setup has two regions with each their own endpoint:
US: o.myobjectstore.us
EU: o.myobjectstore.eu
Global: o.myobjectstore.com _
Bucket _'foo' is in the region US and bucket 'bar' in the EU.
The DNS server will respond like this:
foo.o.myobjectstore.com CNAME foo.o.myobjectstore.us
bar.o.myobjectstore.com CNAME bar.o.myobjectstore.eu
This can be achieved the the "Remote Backend" support from PowerDNS: http://doc.powerdns.com/html/remotebackend.html
To improve performance of the backend for PowerDNS the "Agent" will query the REST Admin API of the RGW.
If that doesn't satisfy the user he/she can put something like Varnish in between to cache HTTP calls and speed up the performance even more.
Work items¶
Coding tasks¶
- Write HTTP backend for PowerDNS (Using the RGW REST Admin API)
- Write a VCL for Varnish
- Implement cache invalidation on bucket removal
Build / release tasks¶
- Create new DEB and RPM package
- Task 2
- Task 3
Documentation tasks¶
- Document on how to set up the remote backend and configure it
- Task 2
- Task 3
Deprecation tasks¶
- Task 1
- Task 2
- Task 3