Ceph

3> 2013-04-01 14:11:17.935826 7fe1427fc700 5 --OSD::tracker- reqid: client.4131.0:27330, seq: 66006, time: 2013-04-01 14:11:17.935826, event: commit_sent, request: osd_sub_op(client.4131.0:273
30 0.31 62635eb1/benchmark_data_plana50_12264_object6533/head//0 [] v 113'479 snapset=0=[]:[] snapc=0=[]) v7
2> 2013-04-01 14:11:17.935871 7fe1427fc700 1 - 10.214.132.34:6804/12731 --> osd.1 10.214.132.34:6801/12729 -- osd_sub_op_reply(client.4131.0:27330 0.31 62635eb1/benchmark_data_plana50_12264_ob
ject6533/head//0 [] ondisk, result = 0) v1 -- ?+0 0x7fe0b009f340
1> 2013-04-01 14:11:17.935922 7fe1427fc700 5 --OSD::tracker- reqid: client.4131.0:27330, seq: 66006, time: 2013-04-01 14:11:17.935922, event: done, request: osd_sub_op(client.4131.0:27330 0.31
62635eb1/benchmark_data_plana50_12264_object6533/head//0 [] v 113'479 snapset=0=[]:[] snapc=0=[]) v7
0> 2013-04-01 14:11:17.953007 7fe1317fa700 1 osd/ReplicatedPG.cc: In function 'void ReplicatedPG::eval_repop(ReplicatedPG::RepGather*)' thread 7fe1317fa700 time 2013-04-01 14:11:17.842792
osd/ReplicatedPG.cc: 3880: FAILED assert(waiting_for_ondisk.begin()>first == repop->v)

ceph version 0.59-753-gdb7a095 (db7a09507e35132fe89e235caaf356ed6db2fde1)
1: (ReplicatedPG::eval_repop(ReplicatedPG::RepGather*)+0x6a0) [0x59a130]
2: (ReplicatedPG::repop_ack(ReplicatedPG::RepGather*, int, int, int, eversion_t)+0x1e4) [0x59b354]
3: (ReplicatedPG::sub_op_modify_reply(std::tr1::shared_ptr<OpRequest>)+0x17c) [0x59ddac]
4: (ReplicatedPG::do_sub_op_reply(std::tr1::shared_ptr<OpRequest>)+0x81) [0x5d4661]
5: (PG::do_request(std::tr1::shared_ptr<OpRequest>)+0x3f0) [0x6a93a0]
6: (OSD::dequeue_op(boost::intrusive_ptr<PG>, std::tr1::shared_ptr<OpRequest>)+0x323) [0x603f43]
7: (OSD::OpWQ::_process(boost::intrusive_ptr<PG>)+0x49b) [0x61ad4b]
8: (ThreadPool::WorkQueueVal<std::pair<boost::intrusive_ptr<PG>, std::tr1::shared_ptr<OpRequest> >, boost::intrusive_ptr<PG> >::_process(boost::intrusive_ptr<PG>, ThreadPool::TPHandle&)+0x31) [0x655
b21]
9: (ThreadPool::WorkQueueVal<std::pair<boost::intrusive_ptr<PG>, std::tr1::shared_ptr<OpRequest> >, boost::intrusive_ptr<PG> >::_void_process(void*, ThreadPool::TPHandle&)+0x9c) [0x655d4c]
10: (ThreadPool::worker(ThreadPool::WorkThread*)+0x4e6) [0x8310b6]
11: (ThreadPool::WorkThread::entry()+0x10) [0x832ee0]
12: (()+0x7e9a) [0x7fe15736be9a]
13: (clone()+0x6d) [0x7fe15576e4bd]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush

ubuntu@teuthology:/var/lib/teuthworker/archive/teuthology-2013-04-01_12:48:27-rados-master-testing-basic/7710/remote