Project

General

Profile

PowerDNS backend for RGW

Summary

When using multiple RGW regions it's hard to direct traffic to the correct region. If a region receives a request which is for a different region it will answer with a 301 HTTP redirect, but this adds extra TCP and HTTP traffic which increases latency.
Wrong configuration on the users end could also lead to additional HTTP traffic on a region since it's only responding with 301s instead of serving objects.

Owners

  • Wido den Hollander (Owner)
  • Name (Affiliation)
  • Name

Interested Parties

  • Name (Affiliation)
  • Name (Affiliation)
  • Name

Current Status

Only an idea, but could be turned into code rather quickly.
Will start with a PoC which won't perform rather well, after that a better version can be written.
Update: POC written and available as Gist on Github: https://gist.github.com/wido/7664859

Detailed Description

A backend for PowerDNS could solve this where the DNS server answers with a different CNAME depending in the bucket in the hostname.
The backend has to be aware of the regions and buckets to be able to direct traffic.
For example, a setup has two regions with each their own endpoint:
US: o.myobjectstore.us
EU: o.myobjectstore.eu
Global: o.myobjectstore.com _
Bucket _'foo'
is in the region US and bucket 'bar' in the EU.
The DNS server will respond like this:
foo.o.myobjectstore.com CNAME foo.o.myobjectstore.us
bar.o.myobjectstore.com CNAME bar.o.myobjectstore.eu
This can be achieved the the "Remote Backend" support from PowerDNS: http://doc.powerdns.com/html/remotebackend.html

To improve performance of the backend for PowerDNS the "Agent" will query the REST Admin API of the RGW.
If that doesn't satisfy the user he/she can put something like Varnish in between to cache HTTP calls and speed up the performance even more.

Work items

Coding tasks

  1. Write HTTP backend for PowerDNS (Using the RGW REST Admin API)
  2. Write a VCL for Varnish
  3. Implement cache invalidation on bucket removal

Build / release tasks

  1. Create new DEB and RPM package
  2. Task 2
  3. Task 3

Documentation tasks

  1. Document on how to set up the remote backend and configure it
  2. Task 2
  3. Task 3

Deprecation tasks

  1. Task 1
  2. Task 2
  3. Task 3