Project

General

Profile

Technical details on the model » History » Version 1

Jessica Mack, 06/01/2015 08:20 PM

1 1 Jessica Mack
h1. Technical details on the model
2 1 Jessica Mack
3 1 Jessica Mack
h4. Some background about replication and erasure coding
4 1 Jessica Mack
 
5 1 Jessica Mack
_"Erasure codes are a superset of replicated and RAID systems."_
6 1 Jessica Mack
The main advantage of erasure codes is that they leverage the statistical stability of large number of components. 
7 1 Jessica Mack
Assuming we have 1 million machines, and 10% are down we can calculate the probability of availability of a block that has 2 replicas using the formula below and we will get two-nines availability:
8 1 Jessica Mack
9 1 Jessica Mack
!Screen_Shot_2014-06-27_at_00.15.48.png!
10 1 Jessica Mack
11 1 Jessica Mack
Instead, we can use erasure coding with the same storage overhead ratio. For instance, we can use k=32, m=32 (in total we will have 64 blocks). The new formula is:
12 1 Jessica Mack
13 1 Jessica Mack
!Screen_Shot_2014-06-27_at_00.54.29.png!
14 1 Jessica Mack
15 1 Jessica Mack
This time with obtain over 8-nines availability. Nice, isn't it?
16 1 Jessica Mack
Source: Weatherspoon, H., Kubiatowicz, J., "Erasure Coding vs. Replication: A quantitative comparison"
17 1 Jessica Mack
18 1 Jessica Mack
h4. Rados
19 1 Jessica Mack
20 1 Jessica Mack
Both replications methods share assumptions:
21 1 Jessica Mack
* Parallel I/O recovery operations.
22 1 Jessica Mack
* Objects are written in the primary OSD of the PG identified by the CRUSH map. The primary daemon contacts other OSDs for replication and recovery purposes.
23 1 Jessica Mack
* Failure rate is constant and follows a Poisson distribution.
24 1 Jessica Mack
 
25 1 Jessica Mack
 
26 1 Jessica Mack
h4. State model (v 0.2)
27 1 Jessica Mack
28 1 Jessica Mack
!simple_state_model_v0.2.jpg!