Ceph

Ran into a wierd case when our configuration management utility had only set ceph_osd_default_pgp_num to a non default value. Since ceph_osd_default_pg_num was still the default of 8; when either creating a pool with rados mkpool or when radosgw started (which in turn creates pools with rados::create_pool) we ran into issues like misdirected client requests, which after a lot of looking at wrong places was finally found out when doing a ceph osd dump

This can be reproduced by setting doing something like

 sudo ceph tell mon.ceph injectargs --osd_pool_default_pgp_num 256

and then creating pool with rados

$ rados mkpool broken_pool
$ ceph osd pool get broken_pool pg_num
pg_num: 8
$ sudo ceph osd pool get broken_pool pgp_num
pgp_num: 256

While ceph osd pool create explicitly catches when setting a pgp_num > pg_num it would be nice if rados mkpool/ pool create had some guard to prevent this from happening