Ceph » RADOS

Current HashIndex::pre_split_folder() use the following snippet to figure the number of levels for split.

leavies /= subs;
while (leavies > 1) {
  ++level;
  leavies = leavies >> 4;
}

The problem is that the number of levels may be far from enough.
For example, we expect 512 PGs, about 1 million files in each PG and about 4096 file in each leave directory. So 9 bits are for PG number, which means top 2 level of directories won't be splitted. And "leavies" here is 244 and "subs" here will be 8. Thus we will only get additional 1 level of splitted directories. The number of files in a PG will be 8*16*4096 (0.5 million) only.

The following snippet will be able to assure enough number of leaves:

  int level  = 0;
  int level_limit = MAX_HASH_LEVEL - dump_num - 1;
  uint64_t actual_leaves = subs;
  while (actual_leaves < leavies && level < level_limit) {
    ++level;
    actual_leaves <<= 4;
  }