Ceph

logs: ubuntu@teuthology:/a/teuthology-2013-05-09_01:00:05-rados-next-testing-basic/9325

2013-05-09T01:37:29.943 INFO:teuthology.task.rados.rados.0.out:Writing plana3332167-39 from 2072826 to 2611791 tid 3 ranges are [0~56,638473~673998,2072826~538965]
2013-05-09T01:37:29.944 INFO:teuthology.task.rados.rados.0.out:1169: oids not in use 497
2013-05-09T01:37:29.944 INFO:teuthology.task.rados.rados.0.out:Reading 22
2013-05-09T01:37:29.944 INFO:teuthology.task.rados.rados.0.out:1170: oids not in use 496
2013-05-09T01:37:29.944 INFO:teuthology.task.rados.rados.0.out:Reading 342
2013-05-09T01:37:29.944 INFO:teuthology.task.rados.rados.0.out:1171: oids not in use 495
2013-05-09T01:37:29.945 INFO:teuthology.task.rados.rados.0.out:Deleting 67 current snap is 0
2013-05-09T01:37:30.131 INFO:teuthology.orchestra.remote:Power off for plana55 completed
2013-05-09T01:37:31.224 INFO:teuthology.task.rados.rados.0.out:incorrect buffer at pos 1146884
2013-05-09T01:37:31.224 INFO:teuthology.task.rados.rados.0.err:Object 342 contents ObjNum: 341 snap: 0 seqnum: 341 prefix: plana3332167-OID: 342 snap 0
2013-05-09T01:37:31.224 INFO:teuthology.task.rados.rados.0.err: corrupt
2013-05-09T01:37:31.225 INFO:teuthology.task.rados.rados.0.err:./test/osd/RadosModel.h: In function 'virtual void ReadOp::_finish(TestOp::CallbackInfo*)' thread 7fe9cdffb700 time 2013-05-09 01:37:46.947876
2013-05-09T01:37:31.225 INFO:teuthology.task.rados.rados.0.err:./test/osd/RadosModel.h: 1053: FAILED assert(0)
2013-05-09T01:37:31.240 INFO:teuthology.task.rados.rados.0.err: ceph version 0.61-160-g5177fcb (5177fcb6bf384d5c9caed9b28f28f656823b3d64)
2013-05-09T01:37:31.240 INFO:teuthology.task.rados.rados.0.err: 1: (ReadOp::_finish(TestOp::CallbackInfo*)+0x13ce) [0x414dbe]
2013-05-09T01:37:31.241 INFO:teuthology.task.rados.rados.0.err: 2: (librados::C_AioComplete::finish(int)+0x1d) [0x7fe9d8003f4d]
2013-05-09T01:37:31.241 INFO:teuthology.task.rados.rados.0.err: 3: (Finisher::finisher_thread_entry()+0x1c0) [0x7fe9d80719d0]
2013-05-09T01:37:31.241 INFO:teuthology.task.rados.rados.0.err: 4: (()+0x7e9a) [0x7fe9d7cb6e9a]
2013-05-09T01:37:31.241 INFO:teuthology.task.rados.rados.0.err: 5: (clone()+0x6d) [0x7fe9d74cdccd]
2013-05-09T01:37:31.242 INFO:teuthology.task.rados.rados.0.err: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2013-05-09T01:37:31.242 INFO:teuthology.task.rados.rados.0.err:terminate called after throwing an instance of 'ceph::FailedAssertion'
2013-05-09T01:37:31.424 ERROR:teuthology.run_tasks:Manager failed: <contextlib.GeneratorContextManager object at 0x320db50>
Traceback (most recent call last):
  File "/var/lib/teuthworker/teuthology-next/teuthology/run_tasks.py", line 45, in run_tasks
    suppress = manager.__exit__(*exc_info)
  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/var/lib/teuthworker/teuthology-next/teuthology/task/rados.py", line 118, in task
    running.get()
  File "/var/lib/teuthworker/teuthology-next/virtualenv/local/lib/python2.7/site-packages/gevent/greenlet.py", line 331, in get
    raise self._exception
CommandCrashedError: Command crashed: 'CEPH_CLIENT_ID=0 /home/ubuntu/cephtest/9325/enable-coredump ceph-coverage /home/ubuntu/cephtest/9325/archive/coverage ceph_test_rados --op read 45 --op write 45 --op delete 10 --op snap_create 0 --op snap_remove 0 --op rollback 0 --op setattr 0 --op rmattr 0 --op watch 0 --max-ops 4000 --objects 500 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0'

ubuntu@teuthology:/a/teuthology-2013-05-09_01:00:05-rados-next-testing-basic/9325$ cat orig.config.yaml 
kernel:
  kdb: true
  sha1: b5b09be30cf99f9c699e825629f02e3bce555d44
machine_type: plana
nuke-on-error: true
overrides:
  ceph:
    conf:
      mon:
        debug mon: 20
        debug ms: 20
        debug paxos: 20
    fs: xfs
    log-whitelist:
    - slow request
    sha1: 6c1e4791782ce2b3e101ee80640d896bcda684de
  s3tests:
    branch: next
  workunit:
    sha1: 6c1e4791782ce2b3e101ee80640d896bcda684de
roles:
- - mon.0
  - mon.1
  - mon.2
  - mds.0
  - client.0
- - osd.0
- - osd.1
- - osd.2
tasks:
- chef: null
- clock.check: null
- install: null
- ceph: null
- thrashosds:
    chance_down: 1.0
    powercycle: true
- rados:
    clients:
    - client.0
    objects: 500
    op_weights:
      delete: 10
      read: 45
      write: 45
    ops: 4000