Project

General

Profile

Adding a proprietary key value store to CEPH as a pluggable module » History » Version 2

Jessica Mack, 07/06/2015 08:55 PM

1 1 Jessica Mack
h1. Adding a proprietary key value store to CEPH as a pluggable module
2 1 Jessica Mack
3 1 Jessica Mack
h3. Summary
4 1 Jessica Mack
5 1 Jessica Mack
Current CEPH , key value store integration works with rocksdb, leveldb and kinetic drives. Adding a mechanisim to integrate any Key Value Store to be added as a CEPH OSD backend honoring the semantics of object store.
6 1 Jessica Mack
7 1 Jessica Mack
h3. Owners
8 1 Jessica Mack
9 1 Jessica Mack
* Varada Kari (Sandisk)
10 1 Jessica Mack
* Somnath Roy(Sandisk)
11 1 Jessica Mack
* Name
12 1 Jessica Mack
13 1 Jessica Mack
h3. Interested Parties
14 1 Jessica Mack
15 1 Jessica Mack
* Danny Al-Gaaf (Deutsche Telekom)
16 1 Jessica Mack
* Name (Affiliation)
17 1 Jessica Mack
* Name
18 1 Jessica Mack
19 1 Jessica Mack
h3. Current Status
20 1 Jessica Mack
 
21 1 Jessica Mack
h3. Detailed Description
22 1 Jessica Mack
23 1 Jessica Mack
The essence of the new scheme is as follows.
24 2 Jessica Mack
# Implement a wrapper around the proprietary KVStore. Let us call it as KVExtension. This is a shared library which implements all interfaces required by CEPH KeyValueStore.
25 2 Jessica Mack
# A new class is derived from KeyValueDB called PropDBStore, which honors the semantics of KeyvalueStore and KeyValueDB. This class acts as mediator between CEPH and KVExtension.  This class transforms bufferlist etc... to const char pointers or strings for the extension to understand.
26 2 Jessica Mack
# PropDBStore, loads (dlopen) the KVExtension during OSD initialization.  Path to the KVExtension can be mentioned in ceph.conf.
27 2 Jessica Mack
# Interfaces that needs to be implemented in KVExtension, which are imported by the PropDBStore are added in a new header called PropDBWrapper.h.  This header contains the signatures for the necessary interfaces like init(), close(), submit_transaction(), get() and get_iterator(). Similarly for Iterator functionality, PropDBIterator.h, which specifies the signatures of seek_to_first (), seek_to_last(), lower_bound() and upper_bound() etc...  PropDBStore includes these headers to import the symbols, using dlsym().
28 2 Jessica Mack
# Choosing the proprietary DB as Backend to the OSD is controlled/managed by config options of the ceph (/etc/ceph/ceph.conf) like rocksdb or leveldb.
29 2 Jessica Mack
# Rest of the existing functionality is not disturbed by this change. Changing the osd backend option will change backend implementation. But this change is not dynamic. The type of the backend should be chosen at osd creation time and osd will continue use that backend till that osd is reformatted again.
30 2 Jessica Mack
# The new KVStore we are trying to integrate works on a raw partition, so we divided the osd drive into two partitions. One partition is given to osd Meta data (super block, fsid etc...), and the other is given to the new db to manage it. OSD partition is now not the entire disk, but 2-4GB which needed for the metadata.
31 1 Jessica Mack
 
32 1 Jessica Mack
want to extend the implementation to make the key value store as dynamically loadable module.
33 1 Jessica Mack
34 1 Jessica Mack
h3. Work items
35 1 Jessica Mack
36 1 Jessica Mack
h4. Coding tasks
37 1 Jessica Mack
38 1 Jessica Mack
# Implement the abstraction to dynamically load the key value db.
39 1 Jessica Mack
# Implement the changes needed in ceph-disk
40 1 Jessica Mack
41 1 Jessica Mack
h4. Build / release tasks
42 1 Jessica Mack
43 1 Jessica Mack
# Task 1
44 1 Jessica Mack
# Task 2
45 1 Jessica Mack
# Task 3
46 1 Jessica Mack
47 1 Jessica Mack
h4. Documentation tasks
48 1 Jessica Mack
49 1 Jessica Mack
# Task 1
50 1 Jessica Mack
# Task 2
51 1 Jessica Mack
# Task 3
52 1 Jessica Mack
53 1 Jessica Mack
h4. Deprecation tasks
54 1 Jessica Mack
55 1 Jessica Mack
# Task 1
56 1 Jessica Mack
# Task 2
57 1 Jessica Mack
# Task 3