Project

General

Profile

Rgw - plugin architecture » History » Version 1

Jessica Mack, 06/07/2015 01:41 AM

1 1 Jessica Mack
h1. Rgw - plugin architecture
2 1 Jessica Mack
3 1 Jessica Mack
h3. Summary
4 1 Jessica Mack
5 1 Jessica Mack
There is a growing amount of functionality in RGW like the S3 API, Swift API, and Keystone Auth. Right now this is not really segregated in the code; we even have a "bool do_swift" in rgw_main::main(). Besides that, we have a cluttered namespace (for instance, we have a suffix '_S3' to all S3 classes and '_SWIFT' to all Swift classes, but Keystone functionality is only available in RGWSwift)
6 1 Jessica Mack
A plugin architecture might solve this.
7 1 Jessica Mack
8 1 Jessica Mack
h3. Owners
9 1 Jessica Mack
10 1 Jessica Mack
* Roald van Loon (roaldvanloon@gmail.com)
11 1 Jessica Mack
* Name (Affiliation)
12 1 Jessica Mack
* Name
13 1 Jessica Mack
14 1 Jessica Mack
h3. Interested Parties
15 1 Jessica Mack
16 1 Jessica Mack
* Roald van Loon (roaldvanloon@gmail.com)
17 1 Jessica Mack
* Name (Affiliation)
18 1 Jessica Mack
* Name
19 1 Jessica Mack
20 1 Jessica Mack
h3. Current Status
21 1 Jessica Mack
22 1 Jessica Mack
I'm currently implementing the proposed solution described below here; https://github.com/roaldvanloon/ceph/tree/wip-rgw-plugin. This is specific for a "AUTH" type plugin for keystone support. For this, I'm also implementing an authentication pipeline mechanism/internal API. This falls a bit out of the scope of this blueprint however.
23 1 Jessica Mack
24 1 Jessica Mack
h3. Detailed Description
25 1 Jessica Mack
26 1 Jessica Mack
I propose to introduce a simple internal API for the plugin system which is build upon a newly @RGWPluginManager@. Other components ("owners") in the RGW can request to "register" plugins of a specific type. For instance, @RGWREST@ might want to register plugins of type @RGW_PLUGIN_TYPE_REST_API@. The API between the RGWPluginManager and the plugin can be very simple and flexible, because the plugin can count on an "owner" of a specific type (e.g. a @RGW_PLUGIN_TYPE_REST_API@ type of plugin will always have an instance of @RGWREST@ as its "owner") and can access it directly.
27 1 Jessica Mack
 
28 1 Jessica Mack
*Terminology*
29 1 Jessica Mack
30 1 Jessica Mack
* _Plugin manager_: instance of RGWPluginManager
31 1 Jessica Mack
* _Owner_: another component within RGW that requests access to plugins through the plugin manager
32 1 Jessica Mack
* _Type_: predefined type of a plugin, for instance RGW_PLUGIN_TYPE_REST_API or RGW_PLUGIN_TYPE_AUTH
33 1 Jessica Mack
* _Loading_: dynamic load the plugin, making it known to the plugin manager
34 1 Jessica Mack
* _Registering_: happens when an owner requests access to plugins; this actually initializes a new instance of the plugin
35 1 Jessica Mack
* _Plugin_: dynamic loadable .so, exporting rgw_plugin_init which is used by the plugin manager
36 1 Jessica Mack
37 1 Jessica Mack
h4. Layout of a plugin
38 1 Jessica Mack
39 1 Jessica Mack
Plugins are in @src/rgw/plugins/[pluginname]@ and are compiled into a .so. They export at least one symbol:
40 1 Jessica Mack
<pre>
41 1 Jessica Mack
extern "C" rgw_plugin* rgw_plugin_init(); 
42 1 Jessica Mack
</pre>
43 1 Jessica Mack
44 1 Jessica Mack
This function returns a new instance of rgw_plugin, a struct defined in rgw_plugin.h as follows;
45 1 Jessica Mack
<pre>
46 1 Jessica Mack
struct rgw_plugin {
47 1 Jessica Mack
  const uint32_t type;
48 1 Jessica Mack
  const bool is_singular;
49 1 Jessica Mack
  int (* init)(rgw_plugin *);
50 1 Jessica Mack
  int (* exit)(rgw_plugin *);
51 1 Jessica Mack
  RGWPluginManager * plugin_manager;
52 1 Jessica Mack
  void * owner;
53 1 Jessica Mack
  CephContext *context;
54 1 Jessica Mack
};
55 1 Jessica Mack
</pre>
56 1 Jessica Mack
57 1 Jessica Mack
It needs to set at least the @type@ of plugin, whether it can be registered multiple times by multiple owners (@is_singular@), and the @init@ and @exit@ functions that the plugin manager can use for each time this plugin gets registered. The plugin manager sets the @plugin_manager@ pointer to itself, so the plugin knows where to find the plugin manager when it needs to. This is useful for instance when a plugin needs/wants access to another plugin.
58 1 Jessica Mack
 
59 1 Jessica Mack
*Step 1 - Loading a plugin*
60 1 Jessica Mack
First, the plugin manager needs to be initialized (this maybe should be a global?). For instance, in @main()@ in @rgw_main.cc;@
61 1 Jessica Mack
<pre>
62 1 Jessica Mack
 RGWPluginManager *pm = new RGWPluginManager(g_ceph_context);
63 1 Jessica Mack
  if (pm->load_plugins() < 0)
64 1 Jessica Mack
    return 1;
65 1 Jessica Mack
</pre>
66 1 Jessica Mack
 
67 1 Jessica Mack
The @load_plugins()@ function will automatically load the plugins that are configured in @OPTION(rgw_load_plugins)@. From this moment on, these plugins can be registered by owners when they want to.
68 1 Jessica Mack
This is a separate call, because a client that will implement @librgw@ in the future might want to load only specific plugins using a @load_plugin()@ method.
69 1 Jessica Mack
 
70 1 Jessica Mack
*Step 2 - Registering a plugin*
71 1 Jessica Mack
For instance, the RGWREST might want to access plugins that supply new APIs (type @RGW_PLUGIN_TYPE_REST_API@). In @main();@
72 1 Jessica Mack
<pre>
73 1 Jessica Mack
RGWREST *rest = new RGWREST(pm);
74 1 Jessica Mack
</pre>
75 1 Jessica Mack
76 1 Jessica Mack
And in the RGWREST constructor;
77 1 Jessica Mack
<pre>
78 1 Jessica Mack
RGWREST::RGWREST(RGWPluginManager *_pm)
79 1 Jessica Mack
: plugin_manager(_pm) {
80 1 Jessica Mack
  plugin_manager->register_plugins(RGW_PLUGIN_TYPE_REST_API, (void *) this);
81 1 Jessica Mack
}
82 1 Jessica Mack
</pre>
83 1 Jessica Mack
84 1 Jessica Mack
It also supplies a void pointer to itself, so the plugin manager can register this instance of @RGWREST@ as the owner. The plugin manager will make a copy of the @rgw_plugin@ definition, and link it to the correct owner (e.g. set the void pointer). Then, it runs the @init()@ function of the plugin to notify the plugin that an owner wants to use this plugin and it needs to initialize.
85 1 Jessica Mack
Within the plugin manager, a list of registered plugins is maintained;
86 1 Jessica Mack
<pre>
87 1 Jessica Mack
map<string, list<rgw_plugin*> > plugin_map;
88 1 Jessica Mack
</pre>
89 1 Jessica Mack
90 1 Jessica Mack
Where the string is the "name" of the plugin and the list is the list of copies of the @rgw_plugin@ struct that the plugin supplied during loading. Of course, when the plugin has @is_singular@ set to true, and this plugin is already registered to another owner, it fails.
91 1 Jessica Mack
Important is that the owner "unregisters" plugins also;
92 1 Jessica Mack
<pre>
93 1 Jessica Mack
RGWREST::~RGWREST() {
94 1 Jessica Mack
  plugin_manager->unregister_plugins(RGW_PLUGIN_TYPE_REST_API, (void *) this);
95 1 Jessica Mack
}
96 1 Jessica Mack
</pre>
97 1 Jessica Mack
98 1 Jessica Mack
This will cause the plugin manager to call the @exit()@ function of the plugin, remove the effective copies of the @rgw_plugin@ plugin definitions from the @plugin_map@, and clean up nicely.
99 1 Jessica Mack
 
100 1 Jessica Mack
h3. Work items
101 1 Jessica Mack
102 1 Jessica Mack
h3. Coding tasks
103 1 Jessica Mack
104 1 Jessica Mack
# Creating the RGWPluginManager
105 1 Jessica Mack
# Modify the rgw_main.cc::main() so that the plugin manager gets initialized.
106 1 Jessica Mack
# For future RGW_PLUGIN_TYPE_REST_API type API support:
107 1 Jessica Mack
## Modify the RGWREST constructor/destructor so that it can be passed an instance of  RGWPluginManager
108 1 Jessica Mack
# For future RGW_PLUGIN_TYPE_AUTH type API support:
109 1 Jessica Mack
## Modify the RGWProcess constructor/destructor so that it can be passed an instance of  RGWPluginManager
110 1 Jessica Mack
111 1 Jessica Mack
h3. Documentation tasks
112 1 Jessica Mack
113 1 Jessica Mack
# Document the RGWPluginManager <-> Plugin API and requirements