Ceph

     0> 2014-05-17 03:23:41.455478 7fac77d56700 -1 osd/ReplicatedPG.cc: In function 'void ReplicatedPG::execute_ctx(ReplicatedPG::OpContext*)' thread 7fac77d56700 time 2014-05-17 03:23:41.215536
osd/ReplicatedPG.cc: 1849: FAILED assert(0 == "out of order op")

 ceph version 0.80-467-g5ace4d1 (5ace4d1af712b6852b27b5046b7df2a66c0f4e51)
 1: (ReplicatedPG::execute_ctx(ReplicatedPG::OpContext*)+0x1dbf) [0x820fbf]
 2: (ReplicatedPG::do_op(std::tr1::shared_ptr<OpRequest>)+0x2d15) [0x82ab15]
 3: (ReplicatedPG::do_request(std::tr1::shared_ptr<OpRequest>, ThreadPool::TPHandle&)+0x692) [0x7c6142]
 4: (OSD::dequeue_op(boost::intrusive_ptr<PG>, std::tr1::shared_ptr<OpRequest>, ThreadPool::TPHandle&)+0x1ca) [0x651e0a]
 5: (OSD::OpWQ::_process(boost::intrusive_ptr<PG>, ThreadPool::TPHandle&)+0x628) [0x6528b8]
 6: (ThreadPool::WorkQueueVal<std::pair<boost::intrusive_ptr<PG>, std::tr1::shared_ptr<OpRequest> >, boost::intrusive_ptr<PG> >::_void_process(void*, ThreadPool::TPHandle&)+0x9c) [0x6831fc]
 7: (ThreadPool::worker(ThreadPool::WorkThread*)+0x4e6) [0xa66456]
 8: (ThreadPool::WorkThread::entry()+0x10) [0xa68260]
 9: (()+0x7e9a) [0x7fac8f602e9a]
 10: (clone()+0x6d) [0x7fac8dbc33fd]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

ubuntu@teuthology:/a/teuthology-2014-05-16_23:00:03-rbd-master-testing-basic-plana/258907