Ceph

This bug popped up while Jim Schutt was trying to reproduce #4999

This was my reply to the assert:

The issue here is that at some point we back off a proposal on one of the PaxosServices and the proposal_timer is scheduled. However, it appears that something was updated leading to a 'Paxos::prepare_bootstrap()' [1], which by setting Paxos as 'going_to_bootstrap' makes 'PaxosService::is_active()' to return false and that assert to fail.

If this is indeed the cause, the easiest solution is fairly simple: just return on propose_pending() if we're going to bootstrap -- then again, this might make us lose the pending state; I guess this warrants another approach.

[1] - For clarity's sake, we use the prepare_bootstrap() approach so that we know that we must not schedule new proposals, but at the same time taking care of any currently queued proposals so that we don't end up losing them.

Jim's log file is attached.