Project

General

Profile

Subtask #7146

Feature #4929: Erasure encoded placement group

implement osd crush rule create-erasure

Added by Loic Dachary over 5 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Monitor
Target version:
-
Start date:
01/14/2014
Due date:
% Done:

100%

Source:
other
Tags:
Backport:
Reviewed:
Affected Versions:
Pull request ID:

Description

Currently, what we need to do is make sure the EC plugin maps onto a
linear array of devices the same way that CRUSH does. For a pyramid code,
the CRUSH rule will be something like

step take root
step choose 3 rack
step choose 5 osd
emit

to get 3 groups of 5 devices as an array of size 15. That means the EC
plugin needs to map onto ranks that go something like

0-3 data
4 local parity
5-8 data
9 local parity
10-11 data
12-13 global parity
14 local parity

(or whatever).

Getting this to line up is a bit fragile, unfortunately. We could make
a plugin method that describes the subgrouping, but even then I'm not
sure how easy it is to programmatically validate that an arbitrary CRUSH
rule will behave well. Maybe it is enough to

- have some way to query the layout of the EC plugin (e.g, 3 groups of 5).
- add a new 'osd crush rule create-pyramid ...' command to supplement
'create-simple'.

and document it well...


Related issues

Related to Ceph - Subtask #7158: EC: flesh out how the ceph tool should be used to manage ec pools and create sub tasks for doing it Resolved 01/15/2014
Related to Ceph - Feature #7238: erasure code : implement LRC plugin Resolved 01/25/2014
Related to RADOS - Feature #7267: erasure code: take advantage of locality for recovery New
Related to Ceph - Subtask #7313: erasure-code: rule create-erasure requires CEPH_FEATURE_CRUSH_V2 Resolved 02/03/2014

History

#1 Updated by Loic Dachary over 5 years ago

  • Status changed from New to In Progress
  • Assignee set to Loic Dachary
  • % Done changed from 0 to 10
  take root
  set choose datacenter 2
  set choose osd 3

   -^000111^-  datacenter
   ^-000111-^

[
    { "plugin": "xor",
      "k": 3,
      "m": 1,
      "item": "datacenter",
      "mapping": "-^000111^-",
    },

    { "plugin": "jerasure",
      "technique": "cauchy_good",
      "k": 6,
      "m": 2,
      "item": "device",
      "mapping": "^-000111-^",
    },
]

  take root
  set choose datacenter 2
  set choose rack 2
  set choose osd 3

   --000^111^^222^333--  rack
   -^000-000--111-111^-  datacenter
   ^-000-000--000-000-^  

[
    { "plugin": "xor",
      "k": 3,
      "m": 1,
      "item": "rack",
      "mapping": "--000^111^^222^333--",
    },

    { "plugin": "xor",
      "k": 6,
      "m": 1,
      "item": "datacenter",
      "mapping": "-^000-000--111-111^-",
    },

    { "plugin": "jerasure",
      "technique": "cauchy_good",
      "k": 12,
      "m": 2,
      "item": "device",
      "mapping": "^-000-000--000-000-^",
    },
]

#2 Updated by Loic Dachary over 5 years ago

> => build 12 data chunks (d1...d12)
> => build 6 RS chunks, distribute  (p1..p6)
> => arrange them as : lp1=(d1,d2,d3,d4,p1,p2) lp2=(d5,d6,d7,d8,p3,p4) lp3=(d9,d10,d11,d12,p5,p6)
> => map 21 stripes to 3 data center as: D1=(d1,d2,d3,d4,p1,p2,lp1) D2=(d5,d6,d7,d8,p3,p4,lp2) D3=(d9,d10,d11,d12,p5,p6,lp3)
> e.g. chunk(0...21) = (d1,d2,d3...lp1,d5,d6,d7...lp2,d9,d10,d11...lp3)

Translates into
  take root
  set choose datacenter 3
  set choose osd 7

#         >      >      >
#       >>|    >>|    >>|
#       |||    |||    |||
#   abcd^^^efgh^^^ijkl^^^
#
#   000000^111111^222222^ datacenter
#   0000^^-0000^^-0000^^- 
#
#
[
    { "plugin": "xor",
      "k": 6,
      "m": 1,
      "item": "datacenter",
      "mapping": "000000^111111^222222^",
    },

    { "plugin": "jerasure",
      "technique": "cauchy_good",
      "k": 12,
      "m": 6,
      "item": "device",
      "mapping": "0000^^-0000^^-0000^^-",
    },
]

#3 Updated by Loic Dachary over 5 years ago

  • % Done changed from 10 to 20

Hi Sage

What if osd crush rule create-pyramid was something like:

  • get the properties of a designated erasure coded pool
  • get the corresponding plugin instance
  • call the add_ruleset() method of the plugin with the crushmap as an argument

In other words, delegate the creation of the ruleset to the plugin entirely instead of getting infos from it and create the ruleset.

What do you think ?

#5 Updated by Loic Dachary over 5 years ago

  • Parent task changed from #4929 to #7266

#6 Updated by Loic Dachary over 5 years ago

  • Subject changed from implement osd crush rule create-pyramid to implement osd crush rule create-erasure
  • Status changed from In Progress to Need Review

#7 Updated by Loic Dachary over 5 years ago

  • % Done changed from 20 to 90

#8 Updated by Loic Dachary over 5 years ago

  • Category changed from OSD to Monitor

#9 Updated by Loic Dachary over 5 years ago

  • Parent task changed from #7266 to #4929

#10 Updated by Loic Dachary over 5 years ago

  • Status changed from Need Review to Resolved
  • % Done changed from 90 to 100
  • translation missing: en.field_remaining_hours set to 0.0

Also available in: Atom PDF