Ceph » rgw

thanks Voja,

i think it's worth raising this for discussion on the dev list. rgw needs the ability to reliably create pools that don't exist. if the new pool requires too many PGs, the cluster may refuse to create them, and this has been very confusing to users historically:

https://tracker.ceph.com/issues/22351
https://tracker.ceph.com/issues/23480

in past discussions with the rados team it was suggested to use very small pg counts for this pool creation, and wait for the autoscaler to fix them up in the background. ideally, the orchestrator would create all of these pools during deployment instead of at rgw runtime, where the orchestrator has much more information about available/target PG sizing

I see. If I understand the links and the concern correctly, the issue may be that trying to create a pool with high PG count could fail and the using the minimum PG count tries to avoid it.

I do not know how things work internally but I am not proposing increasing the initial PG count of the created pool. I am proposing to create the pool with the same PG count as is done now but enable the autoscaler new "bulk" flag for the pool and then later let autoscaler increase the PG count if it can. Perhaps this would have the same end result - I would not know. On my system the pool was created automatically with 32 PG and after setting the "bulk" flag it increased to 1024. I would expect the autoscaler to not try to increase PGs from the initial amount to beyond what is possible.

I can see how especially changing PG counts (as would be done by autoscaler later) could be controversial since it might mean juggling some data around -- granted, the newly created pool should have little data at this point.

As an alternative I propose to mention setting the "bulk" flag in the documentation for those users that use autoscaler. Currently documentation simply says that "PGs may need to be tuned" in https://docs.ceph.com/en/latest/radosgw/pools/#tuning - with no reference to the now enabled by default autoscaler. Since use of autoscaler seems to be encouraged then it might make sense, especially to new users, to mention the "bulk" flag and/or link the users to autoscaler documentation.
If the documentation addition sounds reasonable I could take a stab at a PR for it.

The "bulk" flag is intended for exactly this case - to differentiate between e.g. the rgw data pool and metadata pools that are not expected to need the same level of parallelism. The autoscaler operates within the pg/OSD budget, so it can't hit the issues in the past when pools were created with more pgs. RGW could create the pool with 1 pg and the bulk flag, and the autoscaler would set it to an appropriate value for the cluster size. The orchestrator should use the bulk flag as well, then it doesn't have to worry about exact pg counts.