Ceph

log: ubuntu@teuthology:/a/teuthology-2013-05-13_01:00:04-rados-master-testing-basic/12278

2013-05-13T01:28:13.050 INFO:teuthology.task.rados.rados.0.err:./test/osd/RadosModel.h: In function 'virtual void DeleteOp::_begin()' thread 7f7183f2d780 time 2013-05-13 01:28:32.904617
2013-05-13T01:28:13.051 INFO:teuthology.task.rados.rados.0.err:./test/osd/RadosModel.h: 907: FAILED assert(0)
2013-05-13T01:28:13.051 INFO:teuthology.task.rados.rados.0.err: ceph version 0.61-256-gdb29f49 (db29f49f9eab5fe5bf27d57d623489186ced50ac)
2013-05-13T01:28:13.051 INFO:teuthology.task.rados.rados.0.err: 1: (DeleteOp::_begin()+0x586) [0x415d16]
2013-05-13T01:28:13.051 INFO:teuthology.task.rados.rados.0.err: 2: (RadosTestContext::loop(TestOpGenerator*)+0x9a) [0x40cc2a]
2013-05-13T01:28:13.051 INFO:teuthology.task.rados.rados.0.err: 3: (main()+0xacf) [0x40bc2f]
2013-05-13T01:28:13.051 INFO:teuthology.task.rados.rados.0.err: 4: (__libc_start_main()+0xed) [0x7f71824cc76d]
2013-05-13T01:28:13.052 INFO:teuthology.task.rados.rados.0.err: 5: ceph_test_rados() [0x40bfa1]
2013-05-13T01:28:13.052 INFO:teuthology.task.rados.rados.0.err: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2013-05-13T01:28:13.052 INFO:teuthology.task.rados.rados.0.err:terminate called after throwing an instance of 'ceph::FailedAssertion'
2013-05-13T01:28:13.525 ERROR:teuthology.run_tasks:Manager failed: <contextlib.GeneratorContextManager object at 0x2955110>
Traceback (most recent call last):
  File "/var/lib/teuthworker/teuthology-master/teuthology/run_tasks.py", line 45, in run_tasks
    suppress = manager.__exit__(*exc_info)
  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/var/lib/teuthworker/teuthology-master/teuthology/task/rados.py", line 118, in task
    running.get()
  File "/var/lib/teuthworker/teuthology-master/virtualenv/local/lib/python2.7/site-packages/gevent/greenlet.py", line 331, in get
    raise self._exception
CommandCrashedError: Command crashed: 'CEPH_CLIENT_ID=0 /home/ubuntu/cephtest/12278/enable-coredump ceph-coverage /home/ubuntu/cephtest/12278/archive/coverage ceph_test_rados --op read 45 --op write 45 --op delete 10 --op snap_create 0 --op snap_remove 0 --op rollback 0 --op setattr 0 --op rmattr 0 --op watch 0 --max-ops 4000 --objects 500 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0'

ubuntu@teuthology:/a/teuthology-2013-05-13_01:00:04-rados-master-testing-basic/12278$ cat orig.config.yaml 
kernel:
  kdb: true
  sha1: b5b09be30cf99f9c699e825629f02e3bce555d44
machine_type: plana
nuke-on-error: true
overrides:
  ceph:
    conf:
      global:
        ms inject delay max: 1
        ms inject delay probability: 0.005
        ms inject delay type: osd
        ms inject socket failures: 2500
      mon:
        debug mon: 20
        debug ms: 20
        debug paxos: 20
      osd:
        osd op thread timeout: 60
    fs: btrfs
    log-whitelist:
    - slow request
    sha1: db29f49f9eab5fe5bf27d57d623489186ced50ac
  s3tests:
    branch: master
  workunit:
    sha1: db29f49f9eab5fe5bf27d57d623489186ced50ac
roles:
- - mon.a
  - mon.c
  - osd.0
  - osd.1
  - osd.2
- - mon.b
  - mds.a
  - osd.3
  - osd.4
  - osd.5
  - client.0
tasks:
- chef: null
- clock.check: null
- install: null
- ceph:
    log-whitelist:
    - wrongly marked me down
    - objects unfound and apparently lost
- thrashosds:
    chance_pgnum_grow: 1
    chance_pgpnum_fix: 1
    timeout: 1200
- rados:
    clients:
    - client.0
    objects: 500
    op_weights:
      delete: 10
      read: 45
      write: 45
    ops: 4000