Subtask #7146
closedFeature #4929: Erasure encoded placement group
implement osd crush rule create-erasure
100%
Description
Currently, what we need to do is make sure the EC plugin maps onto a
linear array of devices the same way that CRUSH does. For a pyramid code,
the CRUSH rule will be something like
step take root
step choose 3 rack
step choose 5 osd
emitto get 3 groups of 5 devices as an array of size 15. That means the EC
plugin needs to map onto ranks that go something like
0-3 data
4 local parity
5-8 data
9 local parity
10-11 data
12-13 global parity
14 local parity(or whatever).
Getting this to line up is a bit fragile, unfortunately. We could make
a plugin method that describes the subgrouping, but even then I'm not
sure how easy it is to programmatically validate that an arbitrary CRUSH
rule will behave well. Maybe it is enough to
- have some way to query the layout of the EC plugin (e.g, 3 groups of 5).
- add a new 'osd crush rule create-pyramid ...' command to supplement
'create-simple'.
and document it well...
Updated by Loïc Dachary over 10 years ago
- Status changed from New to In Progress
- Assignee set to Loïc Dachary
- % Done changed from 0 to 10
take root
set choose datacenter 2
set choose osd 3
-^000111^- datacenter
^-000111-^
[
{ "plugin": "xor",
"k": 3,
"m": 1,
"item": "datacenter",
"mapping": "-^000111^-",
},
{ "plugin": "jerasure",
"technique": "cauchy_good",
"k": 6,
"m": 2,
"item": "device",
"mapping": "^-000111-^",
},
]
take root
set choose datacenter 2
set choose rack 2
set choose osd 3
--000^111^^222^333-- rack
-^000-000--111-111^- datacenter
^-000-000--000-000-^
[
{ "plugin": "xor",
"k": 3,
"m": 1,
"item": "rack",
"mapping": "--000^111^^222^333--",
},
{ "plugin": "xor",
"k": 6,
"m": 1,
"item": "datacenter",
"mapping": "-^000-000--111-111^-",
},
{ "plugin": "jerasure",
"technique": "cauchy_good",
"k": 12,
"m": 2,
"item": "device",
"mapping": "^-000-000--000-000-^",
},
]
Updated by Loïc Dachary over 10 years ago
> => build 12 data chunks (d1...d12) > => build 6 RS chunks, distribute (p1..p6) > => arrange them as : lp1=(d1,d2,d3,d4,p1,p2) lp2=(d5,d6,d7,d8,p3,p4) lp3=(d9,d10,d11,d12,p5,p6) > => map 21 stripes to 3 data center as: D1=(d1,d2,d3,d4,p1,p2,lp1) D2=(d5,d6,d7,d8,p3,p4,lp2) D3=(d9,d10,d11,d12,p5,p6,lp3) > e.g. chunk(0...21) = (d1,d2,d3...lp1,d5,d6,d7...lp2,d9,d10,d11...lp3)
Translates into
take root
set choose datacenter 3
set choose osd 7
# > > >
# >>| >>| >>|
# ||| ||| |||
# abcd^^^efgh^^^ijkl^^^
#
# 000000^111111^222222^ datacenter
# 0000^^-0000^^-0000^^-
#
#
[
{ "plugin": "xor",
"k": 6,
"m": 1,
"item": "datacenter",
"mapping": "000000^111111^222222^",
},
{ "plugin": "jerasure",
"technique": "cauchy_good",
"k": 12,
"m": 6,
"item": "device",
"mapping": "0000^^-0000^^-0000^^-",
},
]
Updated by Loïc Dachary over 10 years ago
- % Done changed from 10 to 20
Hi Sage
What if osd crush rule create-pyramid was something like:
- get the properties of a designated erasure coded pool
- get the corresponding plugin instance
- call the add_ruleset() method of the plugin with the crushmap as an argument
In other words, delegate the creation of the ruleset to the plugin entirely instead of getting infos from it and create the ruleset.
What do you think ?
Updated by Loïc Dachary about 10 years ago
- Parent task changed from #4929 to #7266
Updated by Loïc Dachary about 10 years ago
- Subject changed from implement osd crush rule create-pyramid to implement osd crush rule create-erasure
- Status changed from In Progress to Fix Under Review
Updated by Loïc Dachary about 10 years ago
- Parent task changed from #7266 to #4929
Updated by Loïc Dachary about 10 years ago
- Status changed from Fix Under Review to Resolved
- % Done changed from 90 to 100
- Translation missing: en.field_remaining_hours set to 0.0