Ceph

System Environment

os: centos 6.7 x86_64
kernel: 3.10.84
version: ceph-0.94.2

ec-profile:

[guest@localhost src]$ ceph osd erasure-code-profile get ts3ec0603
directory=/usr/lib64/ceph/erasure-code
k=6
m=3
plugin=jerasure
ruleset-failure-domain=host
technique=reed_sol_van
------------------------------------------

hi ceph developers:
i got a big problem that i could not start my ceph cluster after a power failure in my data center (huge CPU cost, and memory out of range, host hang or reboot again and again...), then i have to stop all the osd-servers.

i scan all my disks and build a index that can query all the EC-encoded file's physical path by the object-name.
example: the object "g1vi53.4cb5f6f56638798ea7890.122b5c3ccc65e1f60.0"

10.20.20.191:/ceph/hdd_a45f2520/current/91.27cs0_head/DIR_C/DIR_7/DIR_A/DIR_F/g1vi53.4cb5f6f56638798ea7890.122b5c3ccc65e1f60.0__head_D7F3FA7C__5b_ffffffffffffffff_0 (k1)
10.20.70.59:/ceph/hdd_0547bd97/current/91.27cs1_head/DIR_C/DIR_7/DIR_A/DIR_F/g1vi53.4cb5f6f56638798ea7890.122b5c3ccc65e1f60.0__head_D7F3FA7C__5b_ffffffffffffffff_1 (k2)
10.20.20.192:/ceph/hdd_9271d495/current/91.27cs2_head/DIR_C/DIR_7/DIR_A/DIR_F/g1vi53.4cb5f6f56638798ea7890.122b5c3ccc65e1f60.0__head_D7F3FA7C__5b_ffffffffffffffff_2 (k3)
10.20.70.56:/ceph/hdd_24775a6e/current/91.27cs3_head/DIR_C/DIR_7/DIR_A/DIR_F/g1vi53.4cb5f6f56638798ea7890.122b5c3ccc65e1f60.0__head_D7F3FA7C__5b_ffffffffffffffff_3 (k4)
10.20.70.57:/ceph/hdd_b78e2582/current/91.27cs4_head/DIR_C/DIR_7/DIR_A/DIR_F/g1vi53.4cb5f6f56638798ea7890.122b5c3ccc65e1f60.0__head_D7F3FA7C__5b_ffffffffffffffff_4 (k5)
10.20.70.51:/ceph/hdd_c0bedd33/current/91.27cs5_head/DIR_C/DIR_7/DIR_A/DIR_F/g1vi53.4cb5f6f56638798ea7890.122b5c3ccc65e1f60.0__head_D7F3FA7C__5b_ffffffffffffffff_5 (k6)
10.20.70.54:/ceph/hdd_75b42882/current/91.27cs6_head/DIR_C/DIR_7/DIR_A/DIR_F/g1vi53.4cb5f6f56638798ea7890.122b5c3ccc65e1f60.0__head_D7F3FA7C__5b_ffffffffffffffff_6 (m1)
10.20.70.52:/ceph/hdd_b9c2bfba/current/91.27cs7_head/DIR_C/DIR_7/DIR_A/DIR_F/g1vi53.4cb5f6f56638798ea7890.122b5c3ccc65e1f60.0__head_D7F3FA7C__5b_ffffffffffffffff_7 (m2)
10.20.70.53:/ceph/hdd_1855907b/current/91.27cs8_head/DIR_C/DIR_7/DIR_A/DIR_F/g1vi53.4cb5f6f56638798ea7890.122b5c3ccc65e1f60.0__head_D7F3FA7C__5b_ffffffffffffffff_8 (m3)

then i copy all these files in to a local directory "Coding/" and rename the filename to ec002_k* and ec002_m* order by the physical path's last number of 0~8.

[guest@localhost src]$ ll ./Coding/
rw-r--r- 1 guest guest 118976 Oct 25 17:14 ec002_k1
rw-r--r- 1 guest guest 118976 Oct 25 17:14 ec002_k2
rw-r--r- 1 guest guest 118976 Oct 25 17:14 ec002_k3
rw-r--r- 1 guest guest 118976 Oct 25 17:14 ec002_k4
rw-r--r- 1 guest guest 118976 Oct 25 17:14 ec002_k5
rw-r--r- 1 guest guest 118976 Oct 25 17:14 ec002_k6
rw-r--r- 1 guest guest 118976 Oct 25 17:14 ec002_m1
rw-r--r- 1 guest guest 118976 Oct 25 17:14 ec002_m2
rw-r--r- 1 guest guest 118976 Oct 25 17:14 ec002_m3

then i try to use jerasure command line tools (http://jerasure.org/jerasure-2.0/) to decode the object-file.

[guest@localhost src]$ cat Coding/ec002_meta.txt
ec002
711306 (the real file size)
6 3 8 8 3072
reed_sol_van
0
232

[guest@localhost src]$ decoder ec002
Decoding (MB/sec): inf
De_Total (MB/sec): 22.6118087769

[guest@localhost src]$ ll Coding/ec002_decoded
rw-rw-r- 1 guest guest 711306 Oct 25 17:55 Coding/ec002_decoded

But the decoded file of "ec002_decoded" is not the valid file! i tried to change the ec002_meta.txt with other parameters but still can not decode to the right file.

so. can someone help me recover the EC-encoded object/file from the original files that saved in the disk?