Project

General

Profile

Rbd - ordered crash-consistent write-back caching extension » History » Version 3

Tushar Gohad, 12/03/2015 02:08 PM

1 1 Tushar Gohad
h1. Rbd - crash-consistent ordered write-back caching extension
2 1 Tushar Gohad
3 1 Tushar Gohad
h3. Rationale
4 1 Tushar Gohad
5 1 Tushar Gohad
RBD cache currently supports object-based DRAM caching only, with no ordered write-back support.  Content cacheability is limited by the size of DRAM - proposal here is to extend librbd to support -
6 1 Tushar Gohad
7 2 Tushar Gohad
* A new librbd read cache to support *LBA-based* caching with DRAM/*non-volatile* storage backends
8 3 Tushar Gohad
* An *ordered* *write-back* cache that maintains checkpoints internally (or is structured as a data journal), such that writes that get flushed back to the cluster are always *crash consistent*. Even if one were to lose the client cache entirely, the disk image is still holding a valid file system that looks like it is just a little bit stale [1].  Should have durability characteristics similar to async replication if done right.  
9 2 Tushar Gohad
* External *caching plug-in* interface – kernel and usermode
10 1 Tushar Gohad
11 1 Tushar Gohad
This enhancement has been discussed at the 2015 Ceph hackathon and also at the Tokyo summit with the Ceph/RBD core.  
12 1 Tushar Gohad
13 1 Tushar Gohad
This proposal is based on discussions in [1], [2] with Sage and Josh.  Proof points for client-side caching in [2], [3] and external caching plugin example in Maciej’s CAS POC presentation at the Ceph Performance Weekly Meeting on Nov 25, 2015.
14 1 Tushar Gohad
15 1 Tushar Gohad
Tunable ordered Write-back caching option should provide boost for streaming, log storage, some VDI workloads and eventually consistent databases.  Where write-back is not suitable, a read-only (write-through) option should help improve access latencies - VDI and database workloads for example.
16 1 Tushar Gohad
17 1 Tushar Gohad
h3. Owners
18 1 Tushar Gohad
19 1 Tushar Gohad
* Anjaneya Chagam (Intel)
20 1 Tushar Gohad
* Tushar Gohad (Intel)
21 1 Tushar Gohad
* Stephen Blinick (Intel)
22 1 Tushar Gohad
23 1 Tushar Gohad
h3. Interested Parties
24 1 Tushar Gohad
25 1 Tushar Gohad
* Sage Weil, Josh Durgin
26 1 Tushar Gohad
27 1 Tushar Gohad
h3. Project Phases (current proposal)
28 1 Tushar Gohad
29 1 Tushar Gohad
30 1 Tushar Gohad
* Phase1:  private, LBA-level, read-only cache - plumb in new generic caching layer in librbd for non-volatile storage-backend – possible ObjectCacher replacement.  Read-only cache, designed with scope for write-back caching and pluggability extensions in later phases
31 1 Tushar Gohad
* Phase2: node-local, shared read-only cache. Private ordered/crash-consistent write-back cache
32 1 Tushar Gohad
* Phase3: node-local, shared ordered/crash-consistent write-back cache.  External caching plugins – Intel CAS, dm-cache, etc
33 1 Tushar Gohad
34 1 Tushar Gohad
35 1 Tushar Gohad
h3. References:
36 1 Tushar Gohad
37 1 Tushar Gohad
[1] http://ceph-users.ceph.narkive.com/0LDAkjYy/persistent-write-back-cache
38 1 Tushar Gohad
[2] Ceph Hackathon’15 and Tokyo Summit Discussions on RBD write-back caching
39 1 Tushar Gohad
[3] Whitepaper: database workloads using RBD with dm-cache and Ceph cache tiering