Ceph » CephFS

Original title for the record : "EC pool fails for certain (k,m) combinations for >4MB objs"

Steps to reproduce the error:

git clone https://github.com/ceph/ceph.git
cd ceph
./do_autogen.sh
make
cd src
OSD=5 ./vstart.sh -l -n -X
./ceph osd erasure-code-profile set ecprofile ruleset-failure-domain=osd k=3 m=2 plugin=jerasure
./ceph osd crush rule create-erasure ecruleset ecprofile
./ceph osd pool create ecpool 1 1 erasure ecprofile ecruleset

Once you have the development cluster working you can try:

dd if=/dev/urandom of=./test.dat bs=1MB count=5
./rados -p ecpool put test ./test.dat

And you get the following error:

error putting ecpool/test: (95) Operation not supported

However, the following case works perfectly:

dd if=/dev/urandom of=./test.dat bs=1MB count=4
./rados -p ecpool put test ./test.dat

If instead of (k=3, m=2, OSD=5) you try (k=2, m=2, OSD=4) it works for both 4MB and 5MB objects.

I observed this bug in an Ubuntu Precise machine and in an up-to-date Arch Linux machine.

This same error was first observed by Michael Nelson: http://lists.ceph.com/pipermail/ceph-users-ceph.com/2014-March/028311.html