Project

General

Profile

Pyramid Erasure Code » History » Version 2

Jessica Mack, 06/23/2015 09:24 PM

1 1 Jessica Mack
h1. Pyramid Erasure Code
2 1 Jessica Mack
3 1 Jessica Mack
h3. Summary
4 1 Jessica Mack
5 2 Jessica Mack
!{width:50%}lr.png!
6 1 Jessica Mack
7 1 Jessica Mack
Pyramid codes can be used to avoid using inter datacenter bandwidth when recovering from the loss of a single OSD.
8 1 Jessica Mack
Let say an erasure coded pool is configured so that an object is split in 10 data chunks (X1 to X10) and 4 parity chunks (P1 to P4) and the crush map splits the chunks evenly in two datacenters (X1,X2,X3,X4,X5,P1,P2 in datacenter 1 and X6,X7,X8,X9,X10,P3,P4 in datacenter 2). If X3 is missing, all chunks are needed to recover it. If the primary OSD in charge of scrubbing is X1 (i.e. in datacenter 1), it will need to get X6,X7,X8,X9,X10,P3,P4 from datacenter 2.
9 1 Jessica Mack
One additional parity chunk is calculated  in each datacenter. The S1 chunk is calculated with the chunks from datacenter 1 and is stored in datacenter 1. The S2 chunk is calculated with the chunks from datacenter 2 and is stored in datacenter 2. If X3 is missing and X1 is the primary OSD in charge of scrubbing, it can use X1,X2,X4,X5 and S1 to recover X3 instead of reading from datacenter 2. Reading all chunks is only required when more than one chunk goes missing in the same data center.
10 1 Jessica Mack
11 1 Jessica Mack
h3. Owners
12 1 Jessica Mack
13 1 Jessica Mack
* Loic Dachary <loic@dachary.org>
14 1 Jessica Mack
* Name (Affiliation)
15 1 Jessica Mack
* Name
16 1 Jessica Mack
17 1 Jessica Mack
h3. Interested PartiesEdit section
18 1 Jessica Mack
19 1 Jessica Mack
* Name (Affiliation)
20 1 Jessica Mack
* Name (Affiliation)
21 1 Jessica Mack
* Name
22 1 Jessica Mack
23 1 Jessica Mack
h3. Work items
24 1 Jessica Mack
25 1 Jessica Mack
http://pad.ceph.com/p/cdsgiant-pyramid-erasure-code