PGs per OSD documentation needs clarification
- Desired range for total PGs per OSD (including all Pools and Replicas)
- Impact of empty / non-active Pool's PGs on data distribution and/or Memory/CPU overhead
- My current understanding is that the total PGs per OSD (including all replicas of all pools) should be in the target range of 100 to 200.
(Pool1_pg_num * Pool1_size) + (Pool2_pg_num * Pool2_size) + ... --------------------------------------------------------------- =~ 100 to 200 # of OSDs
- This is to help with ensuring the OSD process' memory and CPU utilization remain in acceptable levels during recovery operations.
- I've also been advised that 500 to 700 total PGs per OSD is generally still OK, but 1000+ total PGs per OSD is considered bad.
- In the last paragraph of section:
50k PGs per OSD is used as an example which would use more resources.. but in reality, I've had experience on a cluster with 9k PGs per OSD that was not able to start and begin stable operations. This example gives an artificially high sense of acceptable norms for PG to OSD count.
- Empty and / or non-active Pools should not be considered helpful toward the overall goal of even data distribution.
Therefore, clusters with only a few active/data containing pools and a number of non-active and / or empty pools may have poor data distribution and thus, 'hot' disks with regard to space utilization.
- They do however still cause CPU and memory overhead as noted above.
- Status changed from New to Closed
This bug has been judged too old to fix. This is because either it is either 1) raised against a version of Ceph prior to Luminous, or 2) just really old, and untouched for so long that it is unlikely nowadays to represent a live documentation concern.
If you think that the closing of this bug is an error, raise another bug of a similar kind. If you think that the matter requires urgent attention, please let Zac Dover know at firstname.lastname@example.org.