Ceph

mira040

2014-09-17 06:36:04.691721 7fabdfc4f700 -1 *** Caught signal (Bus error) **
 in thread 7fabdfc4f700

 ceph version 0.85-767-g9f4c687 (9f4c687288f051d388772b55800258be1b93f85d)
 1: /usr/bin/ceph-mon() [0x89628a]
 2: (()+0xfcb0) [0x7fabeb934cb0]
 3: (()+0x14978e) [0x7fabea33578e]
 4: (()+0x393cd) [0x7fabebb7b3cd]
 5: (leveldb::log::Writer::EmitPhysicalRecord(leveldb::log::RecordType, char const*, unsigned long)+0x100) [0x7fabebb62400]
 6: (leveldb::log::Writer::AddRecord(leveldb::Slice const&)+0x74) [0x7fabebb62554]
 7: (leveldb::DBImpl::Write(leveldb::WriteOptions const&, leveldb::WriteBatch*)+0x1bb) [0x7fabebb5cadb]
 8: (LevelDBStore::submit_transaction_sync(std::tr1::shared_ptr<KeyValueDB::TransactionImpl>)+0x2b) [0x84907b]
 9: (MonitorDBStore::apply_transaction(std::tr1::shared_ptr<MonitorDBStore::Transaction>)+0x16c) [0x542c1c]
 10: (Paxos::handle_begin(MMonPaxos*)+0x548) [0x5ae1c8]
 11: (Paxos::dispatch(PaxosServiceMessage*)+0x24b) [0x5b3b2b]
 12: (Monitor::dispatch(MonSession*, Message*, bool)+0x22c) [0x58abfc]
 13: (Monitor::_ms_dispatch(Message*)+0x208) [0x589168]
 14: (Monitor::ms_dispatch(Message*)+0x32) [0x5a3ec2]
 15: (Messenger::ms_deliver_dispatch(Message*)+0x77) [0x866027]
 16: (DispatchQueue::entry()+0x44a) [0x8631ca]
 17: (DispatchQueue::DispatchThread::entry()+0xd) [0x767d5d]
 18: (()+0x7e9a) [0x7fabeb92ce9a]
 19: (clone()+0x6d) [0x7fabea2dfccd]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

the mon did see the disk fill and shut down. then it was restarted while the disk was full and crashed. i think we should check for the full condition on startup.