Ceph

2014-04-01 22:48:48.290998 7f631bd90700 10 osd.1 285 dequeue_op 0x4c5a1e0 prio 127 cost 0 latency 0.000223 MOSDECSubOpReadReply(3.es2 285 ECSubReadReply(tid=3, attrs_read=0)) v1 pg pg[3.es2( v 277'220 lc 17'80 (0'0,277'220] local-les=285 n=30 ec=8 les/c 285/281 284/284/284) [2,3,1] r=2 lpr=284 pi=237-283/9 rops=1 crt=277'220 lcod 0'0 mlcod 0'0 active+recovering m=9]
2014-04-01 22:48:48.291001 7f631b58f700 10 osd.1 pg_epoch: 285 pg[3.4bs0( v 252'189 lc 13'55 (0'0,252'189] local-les=285 n=29 ec=8 les/c 285/277 284/284/284) [1,3,2] r=0 lpr=284 pi=259-283/6 rops=4 crt=252'189 lcod 0'0 mlcod 0'0 active+recovering m=4] handle_message: MOSDECSubOpReadReply(3.4bs0 285 ECSubReadReply(tid=1, attrs_read=0)) v1
2014-04-01 22:48:48.291034 7f631bd90700 10 osd.1 pg_epoch: 285 pg[3.es2( v 277'220 lc 17'80 (0'0,277'220] local-les=285 n=30 ec=8 les/c 285/281 284/284/284) [2,3,1] r=2 lpr=284 pi=237-283/9 rops=1 crt=277'220 lcod 0'0 mlcod 0'0 active+recovering m=9] handle_message: MOSDECSubOpReadReply(3.es2 285 ECSubReadReply(tid=3, attrs_read=0)) v1
2014-04-01 22:48:48.291031 7f631b58f700 10 osd.1 pg_epoch: 285 pg[3.4bs0( v 252'189 lc 13'55 (0'0,252'189] local-les=285 n=29 ec=8 les/c 285/277 284/284/284) [1,3,2] r=0 lpr=284 pi=259-283/6 rops=4 crt=252'189 lcod 0'0 mlcod 0'0 active+recovering m=4] handle_sub_read_reply: reply ECSubReadReply(tid=1, attrs_read=0)
2014-04-01 22:48:48.291054 7f631bd90700 10 osd.1 pg_epoch: 285 pg[3.es2( v 277'220 lc 17'80 (0'0,277'220] local-les=285 n=30 ec=8 les/c 285/281 284/284/284) [2,3,1] r=2 lpr=284 pi=237-283/9 rops=1 crt=277'220 lcod 0'0 mlcod 0'0 active+recovering m=9] handle_sub_read_reply: reply ECSubReadReply(tid=3, attrs_read=0)
2014-04-01 22:48:48.291084 7f631bd90700 10 osd.1 pg_epoch: 285 pg[3.es2( v 277'220 lc 17'80 (0'0,277'220] local-les=285 n=30 ec=8 les/c 285/281 284/284/284) [2,3,1] r=2 lpr=284 pi=237-283/9 rops=1 crt=277'220 lcod 0'0 mlcod 0'0 active+recovering m=9] handle_sub_read_reply readop complete: ReadOp(tid=3, to_read={e7f90e8e/plana919174-421/head//3=read_request_t(to_read=[0,1052672], need=(2,0),(3,1), want_attrs=1)}, complete={e7f90e8e/plana919174-421/head//3=read_result_t(r=0, errors={}, attrs=1, returned=(0, 1052672, [(2,0),67584, (3,1),67584])}, priority=10, obj_to_source={e7f90e8e/plana919174-421/head//3=(2,0),(3,1)}, source_to_obj={(2,0)=e7f90e8e/plana919174-421/head//3,(3,1)=e7f90e8e/plana919174-421/head//3}, in_progress=)
2014-04-01 22:48:48.291069 7f631b58f700 10 osd.1 pg_epoch: 285 pg[3.4bs0( v 252'189 lc 13'55 (0'0,252'189] local-les=285 n=29 ec=8 les/c 285/277 284/284/284) [1,3,2] r=0 lpr=284 pi=259-283/6 rops=4 crt=252'189 lcod 0'0 mlcod 0'0 active+recovering m=4] handle_sub_read_reply readop complete: ReadOp(tid=1, to_read={825a114b/plana919174-423/head//3=read_request_t(to_read=[0,1052672], need=(2,2),(3,1), want_attrs=1),143ea1cb/plana919174-222/head//3=read_request_t(to_read=[0,1052672], need=(2,2),(3,1), want_attrs=1),3f57e3cb/plana919174-379/head//3=read_request_t(to_read=[0,1052672], need=(2,2),(3,1), want_attrs=1),2d97fbcb/plana919174-385/head//3=read_request_t(to_read=[0,1052672], need=(2,2),(3,1), want_attrs=1)}, complete={825a114b/plana919174-423/head//3=read_result_t(r=0, errors={}, attrs=1, returned=(0, 1052672, [(2,2),526336, (3,1),526336]),143ea1cb/plana919174-222/head//3=read_result_t(r=0, errors={}, attrs=1, returned=(0, 1052672, [(2,2),526336, (3,1),526336]),3f57e3cb/plana919174-379/head//3=read_result_t(r=0, errors={}, attrs=1, returned=(0, 1052672, [(2,2),348160, (3,1),348160]),2d97fbcb/plana919174-385/head//3=read_result_t(r=0, errors={}, attrs=1, returned=(0, 1052672, [(2,2),526336, (3,1),526336])}, priority=10, obj_to_source={825a114b/plana919174-423/head//3=(2,2),(3,1),143ea1cb/plana919174-222/head//3=(2,2),(3,1),3f57e3cb/plana919174-379/head//3=(2,2),(3,1),2d97fbcb/plana919174-385/head//3=(2,2),(3,1)}, source_to_obj={(2,2)=825a114b/plana919174-423/head//3,143ea1cb/plana919174-222/head//3,3f57e3cb/plana919174-379/head//3,2d97fbcb/plana919174-385/head//3,(3,1)=825a114b/plana919174-423/head//3,143ea1cb/plana919174-222/head//3,3f57e3cb/plana919174-379/head//3,2d97fbcb/plana919174-385/head//3}, in_progress=)
2014-04-01 22:48:48.291140 7f631b58f700 10 osd.1 pg_epoch: 285 pg[3.4bs0( v 252'189 lc 13'55 (0'0,252'189] local-les=285 n=29 ec=8 les/c 285/277 284/284/284) [1,3,2] r=0 lpr=284 pi=259-283/6 rops=4 crt=252'189 lcod 0'0 mlcod 0'0 active+recovering m=4] handle_recovery_read_complete: returned 825a114b/plana919174-423/head//3 (0, 1052672, [(2,2),526336, (3,1),526336])
2014-04-01 22:48:48.291137 7f631bd90700 10 osd.1 pg_epoch: 285 pg[3.es2( v 277'220 lc 17'80 (0'0,277'220] local-les=285 n=30 ec=8 les/c 285/281 284/284/284) [2,3,1] r=2 lpr=284 pi=237-283/9 rops=1 crt=277'220 lcod 0'0 mlcod 0'0 active+recovering m=9] handle_recovery_read_complete: returned e7f90e8e/plana919174-421/head//3 (0, 1052672, [(2,0),67584, (3,1),67584])
2014-04-01 22:48:48.291176 7f631b58f700 10 osd.1 pg_epoch: 285 pg[3.4bs0( v 252'189 lc 13'55 (0'0,252'189] local-les=285 n=29 ec=8 les/c 285/277 284/284/284) [1,3,2] r=0 lpr=284 pi=259-283/6 rops=4 crt=252'189 lcod 0'0 mlcod 0'0 active+recovering m=4] handle_recovery_read_complete: [1,526336, 2,526336]
2014-04-01 22:48:48.291177 7f631bd90700 10 osd.1 pg_epoch: 285 pg[3.es2( v 277'220 lc 17'80 (0'0,277'220] local-les=285 n=30 ec=8 les/c 285/281 284/284/284) [2,3,1] r=2 lpr=284 pi=237-283/9 rops=1 crt=277'220 lcod 0'0 mlcod 0'0 active+recovering m=9] handle_recovery_read_complete: [0,67584, 1,67584]
2014-04-01 22:48:48.291863 7f631bd90700 -1 *** Caught signal (Segmentation fault) **
 in thread 7f631bd90700

 ceph version 0.78-495-g5c9b8a2 (5c9b8a271588e39fe6e77bd7a88bcf6b535e1d3e)
 1: ceph-osd() [0x9825ba]
 2: (()+0xfcb0) [0x7f6330c9ccb0]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

ubuntu@teuthology:/var/lib/teuthworker/archive/teuthology-2014-04-01_02:30:19-rados-firefly-distro-basic-plana/158113