Project

General

Profile

Feature #4929

Updated by Loïc Dachary over 10 years ago

This is the _home page_ of erasure coding implementation in Ceph. The description should contain links to all the background information used by the current developers and required for a new developer to jump in. The sub tasks are a complete inventory of the working currently going on and each of them contains links to the corresponding github pull requests or work in progress.

Documentation:

* "Erasure code internal documentation":http://ceph.com/docs/master/dev/osd_internals/erasure_coding/
* "FAST Erasure Code tutorial":http://web.eecs.utk.edu/~plank/plank/papers/2013-02-11-FAST-Tutorial.pdf

"Emperor Ceph summit":http://wiki.ceph.com/01Planning/CDS/Emperor:

* http://wiki.ceph.com/01Planning/02Blueprints/Emperor/Erasure_coded_storage_backend_%28step_2%29
* Introduction of the second day, "Sage explains the Big Rados Features, including erasure code":http://youtube.com/watch?v=-K8bSHx7zJ0&t=19m14s
Sage : _... The big one is erasure coding which is great to have more efficient storage and for tiering. It's good for performances and overall for the cost / performance ratio. ... Erasure coding : the motivation here is that erasure coding is a different way to handle data redundancy using much less overall space but you actually get higher overall data durability so you have a lower probabily of losing data when machine dies, disk dies and so forth. Work has already started in this area by Loic and that's been great so over this next two cycles we're going to be putting Inktank resources and our efforts as well : there is a lot of code refactoring, moving around, making sure that it fits nicely into the structure of rados, so we're very excited to get started on that work. ..._

Dumpling Ceph summit documents:
* http://wiki.ceph.com/01Planning/02Blueprints/Dumpling/Erasure_encoding_as_a_storage_backend
* http://pad.ceph.com/p/Erasure_encoding_as_a_storage_backend

Discussions:

* "PGBackend proposal":http://marc.info/?t=137537535900002&r=1&w=4
* "Erasure code library summary":http://thread.gmane.org/gmane.comp.file-systems.ceph.devel/15649
* "Comments on Ceph distributed parity implementation":http://thread.gmane.org/gmane.comp.file-systems.ceph.devel/15580
* "Erasure coding implementation : high level description":http://thread.gmane.org/gmane.comp.file-systems.ceph.devel/15804
* "Erasure coding library API":http://thread.gmane.org/gmane.comp.file-systems.ceph.devel/15848
* "CEPH Erasure Encoding + OSD Scalability":http://marc.info/?t=137294343200001&r=1&w=2

Alpha testers:

First testing stage is by running teuthology tests independently, as soon as they include Erasure Code tests.

* Université de Nantes - Yann
* CERN - Andreas
* Rackspace - Darren
* Telekom - Michael
* Cloudapt - Mike
* HTS - Samuel

Back