1E - RADOS Gateway refactor into library internal APIs » History » Version 2
Jessica Mack, 06/22/2015 04:38 AM
1 | 1 | Jessica Mack | h1. 1E - RADOS Gateway refactor into library internal APIs |
---|---|---|---|
2 | |||
3 | h3. Live Pad |
||
4 | |||
5 | 2 | Jessica Mack | The live pad can be found here: "[pad]":http://pad.ceph.com/p/RADOS_Gateway_refactor |
6 | 1 | Jessica Mack | |
7 | h3. Summit Snapshot |
||
8 | |||
9 | Coding tasks |
||
10 | # [Wido] Draft librgw API (informed by current rgw, and APIs of initial users that will utilize it... e.g. apache module interface, nginx plugin api, etc.) |
||
11 | # > Highest level API, where you just do GET and PUT |
||
12 | # [Yehuda?] Refactor rgw as need to match that interface |
||
13 | # [Gary] Repackage rgw bits into librgw (C++, C) |
||
14 | # > Lower level API for listing buckets, creating objects, etc, etc |
||
15 | # [Wido?] Python bindings for librgw |
||
16 | # Write rgwhttp (Python? Twisted Web server?) |
||
17 | # Write a MQ reader for Varnish (or any other caching reverse proxy) |
||
18 | |||
19 | Build / release tasks |
||
20 | # Package librgw |
||
21 | # Package rgwhttp (Own Git repository?) |
||
22 | # Package the MQ reader |
||
23 | |||
24 | Documentation tasks |
||
25 | # Document the API of librgw |
||
26 | # Document rgwhttp |
||
27 | # Write new documentation on how to set everything up |
||
28 | |||
29 | Deprecation tasks |
||
30 | # Remove RGW from the main repository |
||
31 | # Remove any old documentation |
||
32 | |||
33 | Other use cases: |
||
34 | # storing large objects with atomic replacement, collection/bucket indeces |
||
35 | # object expiration agents? (instead of embedding in radosgw daemon) |
||
36 | # librgw could provide a migration path from Swift to native RADOS access without proxy server/gateway bottlenecks |
||
37 | |||
38 | |||
39 | OT: other S3/Swift API features we are missing: |
||
40 | # versioning |
||
41 | # object expiration (delete after, delete at) |
||
42 | # Swift temp URLs, #3454 |
||
43 | # temporary security credentials |
||
44 | |||
45 | |||
46 | Comments |
||
47 | 1. Why integrate with Varnish specifically? This doesn't seem very Ceph-related. It's basically an entirely different layer. |
||
48 | For example, we have multiple layers of Varnish in our infrastructure in front of Apache/radosgw and we handle purges already via pre-existing mechanisms (multicast HTCP). |
||
49 | I'm guessing most people have their own way of handling this (CDNs etc.). For some people this might just mean TTLs, others might care for instant purges. |
||
50 | |||
51 | -> Having a "hooks" system where radosgw invokes something e.g. on PUT/POST/DELETE operations could potentially solve this use case. |
||
52 | |||
53 | * -> We probably don't want this in librgw in the beginning, maybe in a later version. Could be handled by the HTTP server as well. |
||
54 | * perhaps a unix socket or fork/exec |
||
55 | |||
56 | 2. Python isn't very good for performance and it's one of Swift's bottlenecks. This would seem a step backwards. |
||
57 | -> It could be any language when librgw exists |
||
58 | |||
59 | 3. Would it make sense to support WSGI middlewares in the Python webserver to provide an easier migration path from Swift? |