Ceph

2013-02-16 16:23:37.867186 7f97a659e700 -1 mon/PaxosService.cc: In function 'void PaxosService::propose_pending()' thread 7f97a659e700 time 2013-02-16 16:23:37.861106
mon/PaxosService.cc: 127: FAILED assert(have_pending)

 ceph version 0.56-789-g980708b (980708b0be77be2758f126b036a8e804ff231e38)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x95) [0x916609]
 2: (PaxosService::propose_pending()+0x143) [0x76ec25]
 3: (AuthMonitor::check_rotate()+0x17c) [0x802a58]
 4: (AuthMonitor::tick()+0x160) [0x802c02]
 5: (Monitor::tick()+0x126) [0x71323a]
 6: (C_Mon_Tick::finish(int)+0x1f) [0x72aba3]
 7: (SafeTimer::timer_thread()+0x350) [0x90cc34]
 8: (SafeTimerThread::entry()+0x1c) [0x90e08a]
 9: (Thread::_entry_func(void*)+0x23) [0x909cbd]
 10: (()+0x7e9a) [0x7f97aa8f6e9a]
 11: (clone()+0x6d) [0x7f97a90af4bd]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

The cause: The Paxos Service's criteria for 'active' didn't include the possibility of Paxos itself proposing a value -- instead, it was just assuming that only Paxos services would ever propose a value. This caused a service to assume it was 'active', thus updated from paxos and tried to propose a value.