Ceph » mgr

In the balancer/module.py we have

            left = max_optimizations
            ...
            available = left - (num_pg - num_pg_active_clean)
            did = plan.osdmap.calc_pg_upmaps(inc, max_deviation, available, [pool])

The intention is not to balance PGs which are scrubbing, but this calculation of `available` means that if you have more than 10 PGs not in "active+clean" state (even out of thousands), there will be no balancing; instead you will get:

2020-11-20T11:28:28.966+0100 7fcee27a0700 -1 calc_pg_upmaps abort due to max <= 0

For example, in our cluster right now:

             8493 active+clean
             20   active+remapped+backfilling

and with `mgr/balancer/upmap_max_optimizations = 20` there is no more balancing triggered (max = 0).

This behaviour was introduced in 86444dbfe5d8478530182116507d034d9e180f5e. It wasn't backported to nautilus which is why we didn't notice until now.

I will send a PR with this fix, which I think was the intention here:

diff --git a/src/pybind/mgr/balancer/module.py b/src/pybind/mgr/balancer/module.py
index 5fffe01dcb..dd9fac46ec 100644
--- a/src/pybind/mgr/balancer/module.py
+++ b/src/pybind/mgr/balancer/module.py
@@ -1023,7 +1023,7 @@ class Module(MgrModule):
                     if s['state_name'] == 'active+clean':
                         num_pg_active_clean += s['count']
                         break
-            available = left - (num_pg - num_pg_active_clean)
+            available = min(left, num_pg_active_clean)
             did = plan.osdmap.calc_pg_upmaps(inc, max_deviation, available, [pool])
             total_did += did
             left -= did