Project

General

Profile

Subtask #5878

Feature #4929: Erasure encoded placement group

Subtask #5877: Plugable erasure code library

erasure plugin mechanism and abstract API

Added by Loïc Dachary over 10 years ago. Updated about 10 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
OSD
Target version:
-
% Done:

100%

Source:
Development
Tags:
Backport:
Reviewed:
Affected Versions:
Pull request ID:

Description

work in progress

The abstract API is provided by dynamically loaded plugins.
  • Abstract Interface
  • Plugin loader
  • unit tests
    • in the test file load the plugin from source
    • simple minded xor based erasure code plugin that only supports M=2,K=1
    • the plugin can be directed to trigger errors for tests purposes
    • implement the test plugin to be the basis of a documented example to follow when implementing a new plugin

Discussions


Related issues

Related to Ceph - Feature #6000: EC: [link] erasure plugin mechanism and abstract API Resolved 08/15/2013

Associated revisions

Revision dde21bdf (diff)
Added by Loïc Dachary over 10 years ago

ErasureCode: abstract interface

The erasure coded pool relies on this abstract interface to encode and
decode the chunks stored in the OSD. It has been designed to be
generic enough to accomodate the libraries and algorithms that are
most likely to be used. It does not claim to be universal.

http://tracker.ceph.com/issues/5878 refs #5878

Signed-off-by: Loic Dachary <>

Revision 640f2f21 (diff)
Added by Loïc Dachary over 10 years ago

ErasureCode: example implementation : K=2 M=1

An erasure code implementation designed for tests. Although it is fully
functional and could be used on actual data, it is mainly provided for
testing purposes. It splits data in two, computes an XOR parity and
can sustain the loss of one chunk.

The constructor will usleep(3) for parameters["usleep"] microseconds
so that the caller can create race conditions.

http://tracker.ceph.com/issues/5878 refs #5878

Signed-off-by: Loic Dachary <>

Revision b61369c8 (diff)
Added by Loïc Dachary over 10 years ago

ErasureCodePlugin: plugin interface

When dynamically loaded, a plugin is expected to define

int __erasure_code_init(char *plugin_name);

When called, it is responsible for registering an ErasureCodePlugin
derived object that provides a factory method from which the concrete
implementation of the ErasureCodeInterface object can be generated:

virtual int factory(const map&lt;std::string,std::string&gt; &parameters,
ErasureCodeInterfaceRef *erasure_code) {
*erasure_code = ErasureCodeInterfaceRef(new ErasureCodeExample(parameters));
return 0;
}

The plugin instance contains the library data member which is used to
store the handle of the shared library. It is opaque to the plugin.

http://tracker.ceph.com/issues/5878 refs #5878

Signed-off-by: Loic Dachary <>

Revision 916901f9 (diff)
Added by Loïc Dachary over 10 years ago

ErasureCodePlugin: plugin registry

A ErasureCodePluginRegistry singleton holds all erasure plugin objects
derived from ErasureCodePlugin and dlopen(2) handles for the lifetime
of the OSD and is cleaned up by the destructor.

The registry has a single entry point ( method factory ) and should
be used as follows:

map&lt;std::string,std::string&gt; parameters;
parameters["directory"] = "/usr/lib/ceph/erasure-code";
ErasureCodeInterfaceRef erasure_code;
ErasureCodePluginRegistry &instance = ErasureCodePluginRegistry::instance();
instance.factory("jerasure", parameters, &erasure_code));

If the plugin requested ( "jerasure" in the example above ) is not
found in the plugins data member, the load method is called and will:

  • dlopen(parameters["erasure-code-directory"] + "jerasure")
  • f = dlsym("__erasure_code_init")
  • f("jerasure")
  • check that it registered "jerasure"

The plugin is expected to do something like

instance.add(plugin_name, new ErasureCodePluginJerasure());

to register itself.

The factory method is protected with a Mutex to avoid race
conditions when using the same plugin from two threads.

The erasure_codelib_LTLIBRARIES variable is added to the Makefile
and the plugins are expected to add themselves and be installed
in the $(libdir)/erasure-code

http://tracker.ceph.com/issues/5878 refs #5878

Signed-off-by: Loic Dachary <>

Revision 39217947 (diff)
Added by Loïc Dachary over 10 years ago

ErasureCodePlugin: plugin registry tests and example

libec_example.la is a fully functional plugin based on
ErasureCodeExample to test the ErasureCodePlugin abstract
interface. It is dynamically loaded to test the
ErasureCodePluginRegistry implementation.

Although the plugin is built in the test directory, it will be
installed. noinst_LTLIBRARIES won't build the shared library, only the
static version which is not suitable for testing.

http://tracker.ceph.com/issues/5878 refs #5878

Signed-off-by: Loic Dachary <>

History

#1 Updated by Loïc Dachary over 10 years ago

  • Category set to OSD

#2 Updated by Loïc Dachary over 10 years ago

  • Source changed from other to Development

#3 Updated by Loïc Dachary over 10 years ago

  • Description updated (diff)

#4 Updated by Loïc Dachary over 10 years ago

  • Description updated (diff)

#5 Updated by Loïc Dachary over 10 years ago

  • Description updated (diff)

#6 Updated by Loïc Dachary over 10 years ago

  • Assignee set to Loïc Dachary

#7 Updated by Loïc Dachary over 10 years ago

  • Status changed from New to In Progress

#8 Updated by Loïc Dachary over 10 years ago

  • Description updated (diff)

#9 Updated by Loïc Dachary over 10 years ago

  • Description updated (diff)

#10 Updated by Loïc Dachary over 10 years ago

  • Description updated (diff)

#11 Updated by Loïc Dachary over 10 years ago

  • Description updated (diff)

#12 Updated by Loïc Dachary over 10 years ago

  • Description updated (diff)

#13 Updated by Loïc Dachary over 10 years ago

  • Description updated (diff)

#14 Updated by Loïc Dachary over 10 years ago

  • Description updated (diff)

#15 Updated by Loïc Dachary over 10 years ago

  • Description updated (diff)

#16 Updated by Loïc Dachary over 10 years ago

  • Description updated (diff)

#17 Updated by Loïc Dachary over 10 years ago

  • Description updated (diff)

#18 Updated by Wido den Hollander over 10 years ago

I haven't looked at it in-depth, but one thing I noticed is that Reed-Solomon is always spelled with the first two letters capatalized.

Will this be case sensitve? I would suggest not, since that will confuse users. I personally dislike CLI tools which are case sensitive for key=value parameters. I always prefer lowercase.

#19 Updated by Loïc Dachary over 10 years ago

Wido den Hollander wrote:

Will this be case sensitve? I would suggest not, since that will confuse users. I personally dislike CLI tools which are case sensitive for key=value parameters. I always prefer lowercase.

In this case Reed-Solomon is the value interpreted by the plugin and I agree with you : it would be better if case is ignored :-)

#20 Updated by Loïc Dachary over 10 years ago

  • Description updated (diff)

#21 Updated by Loïc Dachary over 10 years ago

  • Description updated (diff)

#22 Updated by Loïc Dachary over 10 years ago

  • % Done changed from 0 to 50

#23 Updated by Loïc Dachary over 10 years ago

  • Description updated (diff)

#24 Updated by Loïc Dachary over 10 years ago

  • Description updated (diff)
  • Status changed from In Progress to Fix Under Review
  • % Done changed from 50 to 90

#25 Updated by Loïc Dachary over 10 years ago

  • Description updated (diff)

#26 Updated by Loïc Dachary over 10 years ago

  • Status changed from Fix Under Review to Resolved
  • % Done changed from 90 to 100
  • translation missing: en.field_remaining_hours set to 0.00

#27 Updated by Loïc Dachary about 10 years ago

  • Estimated time set to 0.00 h

Also available in: Atom PDF