Ceph

Hi,

we are experiencing that our OSDS are very unbalanced, some of them are filled to 50% and some others go up to 93%, I have attached the details (ceph osd tree/ceph osd df/crushmap)

For the cluster story, it have started we 3 hosts and 9 OSDs with firefly version and now it have 50 OSDs across 13 hosts with hammer (0.94.6) on ubuntu trusty. (We have used giant for while between)

The weight of each OSDs was ~ 1.0 for 1T, but we started to have trouble about data repartition, we got our first OSD full and then start reducing the weight to free space on full OSDs.

After some reading, we discover that we should enable the tunable "straw_calc_version = 1" to fix the situation, we do, got a big data rebalancing.

But the data repartition was not better.

So we try to use straw2 instead of straw1. The data have moved a lot too but we are still experiencing a bad data repartition.

Cheers,