CDM 06-MAR-2024 » History » Revision 10
Revision 9 (Brad Hubbard, 03/07/2024 02:57 AM) → Revision 10/12 (Brad Hubbard, 03/07/2024 02:58 AM)
h1. CDM 06-MAR-2024
- [Brad] Initial discussions around protecting the manager -> mon queue from overload by rogue modules
* Get a better understanding of what this throttle ("throttle-mgr_mon_messsages") represents (and why it seems to have any relation to the volume plugin)
* Try a test where the volumes plugin is artificially *hung*
** Fail the fs (cephfs fail [fsname]), spawn a bunch of clients that send messages to the volumes plugin and see if we see this scenario happening. Something like `ceph fs subvolume create <fs_name> subvol.XYZ`.
- [Patrick/Venky; if we're available] ceph-mgr throttles hit after module (volumes) finisher thread queue fills up with commands. This causes messenger threads to throttle commands/queries from other clients. Idea for discussion: throttle clients by connection to ensure fairness.
* Allocate each client a number of slots in the throttle and use this to ensure some level of fairness in servicing each module
* Note that we may have a far better understanding of how to resolve this once we have a far better understanding of the issue described in item 1 which is hitherto not really comprehensively understood.