Project

General

Profile

Osd - create backend for seagate kinetic » History » Version 1

Jessica Mack, 06/30/2015 11:51 PM

1 1 Jessica Mack
h1. Osd - create backend for seagate kinetic
2 1 Jessica Mack
3 1 Jessica Mack
h3. Summary
4 1 Jessica Mack
5 1 Jessica Mack
Create a version of KeyValueStore that can run on a seagate kinetic drive
6 1 Jessica Mack
7 1 Jessica Mack
h3. Owners
8 1 Jessica Mack
9 1 Jessica Mack
* Sage Weil (Inktank)
10 1 Jessica Mack
* Name (Affiliation)
11 1 Jessica Mack
* Name
12 1 Jessica Mack
13 1 Jessica Mack
h3. Interested Parties
14 1 Jessica Mack
15 1 Jessica Mack
* Haomai Wang (UnitedStack)
16 1 Jessica Mack
* Danny Al-Gaaf (Deutsche Telekom AG)
17 1 Jessica Mack
* Name (Affiliation)
18 1 Jessica Mack
* Name
19 1 Jessica Mack
20 1 Jessica Mack
h3. Current Status
21 1 Jessica Mack
22 1 Jessica Mack
* a KeyValueStore backend exists that is based on GenericObjectMap and the generic KeyValueDB
23 1 Jessica Mack
* a C/C++ kinetic API will be available shortly
24 1 Jessica Mack
* the k/v backends are key-granularity.  byte object data is striped across keys of some size, and small updates require a read/modify/write.
25 1 Jessica Mack
26 1 Jessica Mack
h3. Detailed Description
27 1 Jessica Mack
28 1 Jessica Mack
There are several challenges to run Ceph backed by kinetic:
29 1 Jessica Mack
* the kinetic API does not expose a transaction interface.  for the first-pass prototype we will ignore this.  for a second pass, we will either do some form of write-ahead logging, come up with something clever that fits well with GenericObjectMap, or convince the kinetic folks to expose transactions through their API.  Interestingly, other backends that do have some 'batch put' capabilities still limit the size of the transaction, so adding a fallback that writes the entire transaction and then applies it and then setting the supported transaction size to 1 should capture the degenerate case.  
30 1 Jessica Mack
31 1 Jessica Mack
h3. Work items
32 1 Jessica Mack
33 1 Jessica Mack
h4. Coding tasks
34 1 Jessica Mack
35 1 Jessica Mack
# make KeyValueDB wrapper for libkinetic
36 1 Jessica Mack
# make a transactional fallback
37 1 Jessica Mack
## a KeyValueDB implementation that sits on top of a NonTransactionalKeyValueDB which includes a batch_put() operation and a get_max_batch_size()
38 1 Jessica Mack
# build/adapt a simple caching layer that sits between KeyValueDB and GenericObjectMap
39 1 Jessica Mack
## or one that is integrated into GenericObjectMap to avoid encode/decode overheads
40 1 Jessica Mack
# evaluate under object and block workloads
41 1 Jessica Mack
42 1 Jessica Mack
h4. Build / release tasks
43 1 Jessica Mack
44 1 Jessica Mack
# include libkinetic packages in ceph repos
45 1 Jessica Mack
## build as necessary for supported platforms
46 1 Jessica Mack
# include support for simulator in vstart or other developer-friendly tooling
47 1 Jessica Mack
48 1 Jessica Mack
h4. Documentation tasks
49 1 Jessica Mack
50 1 Jessica Mack
# blog