Ceph

There are some options that will affect cluster-wide behavior.

Having all the monitors in the quorum using the leader's options will make for more consistent behavior and lower tendency to weird, hard to debug behavior from long-forgotten or creatively set options on some monitors. Will also ease reporting some options have been set (see #7784 for an example).

This will require identifying the options that fall under this category, and get some config watchers on each monitor component to disseminate new options and report on them, or to tell the user those options must be set through the leader, and have the leader disseminate those options to all the quorum members. Leveraging paxos for this would actually be cool and a straightforward way to guarantee that all monitors get the same config at the same point in time.