Ceph » devops

This maybe similar to #9185

Logs are in http://qa-proxy.ceph.com/teuthology/teuthology-2014-08-21_08:29:18-upgrade:dumpling-dumpling-distro-basic-vps/439339/

If you look at this:

2014-08-21T09:11:03.205 INFO:teuthology.orchestra.run.vpm063.stderr:testPUT (test.functional.tests.TestAccount) ... ok
2014-08-21T09:11:07.345 INFO:tasks.ceph.osd.1:Stopped
2014-08-21T09:11:12.346 INFO:tasks.thrashosds.thrasher:in_osds:  [0, 2, 4, 5, 3]  out_osds:  [1] dead_osds:  [1] live_osds:  [0, 3, 2, 5, 4]
2014-08-21T09:11:12.346 INFO:tasks.thrashosds.thrasher:choose_action: min_in 3 min_out 0 min_live 2 min_dead 0
2014-08-21T09:11:12.347 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [0, 3, 2, 5, 4]
2014-08-21T09:11:18.347 INFO:tasks.ceph.osd.2:Stopped
2014-08-21T09:11:23.347 INFO:tasks.thrashosds.thrasher:in_osds:  [0, 2, 4, 5, 3]  out_osds:  [1] dead_osds:  [1, 2] live_osds:  [0, 3, 5, 4]
2014-08-21T09:11:23.347 INFO:tasks.thrashosds.thrasher:choose_action: min_in 3 min_out 0 min_live 2 min_dead 0
2014-08-21T09:11:23.348 INFO:tasks.thrashosds.thrasher:inject_pause on 4
2014-08-21T09:11:23.348 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3
2014-08-21T09:11:23.348 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False
2014-08-21T09:11:23.349 INFO:teuthology.orchestra.run.vpm063:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3'
2014-08-21T09:11:28.449 INFO:tasks.thrashosds.thrasher:in_osds:  [0, 2, 4, 5, 3]  out_osds:  [1] dead_osds:  [1, 2] live_osds:  [0, 3, 5, 4]
2014-08-21T09:11:28.449 INFO:tasks.thrashosds.thrasher:choose_action: min_in 3 min_out 0 min_live 2 min_dead 0
2014-08-21T09:11:28.449 INFO:tasks.thrashosds.thrasher:fixing pg num pool .rgw.buckets
2014-08-21T09:11:28.450 INFO:teuthology.orchestra.run.vpm064:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph pg dump --format=json'
2014-08-21T09:11:28.623 INFO:teuthology.orchestra.run.vpm064.stderr:dumped all in format json
2014-08-21T09:11:28.668 INFO:teuthology.orchestra.run.vpm064:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph osd pool set .rgw.buckets pgp_num 8'
2014-08-21T09:11:28.979 INFO:teuthology.orchestra.run.vpm064.stderr:set pool 12 pgp_num to 8
2014-08-21T09:11:33.987 INFO:tasks.thrashosds.thrasher:in_osds:  [0, 2, 4, 5, 3]  out_osds:  [1] dead_osds:  [1, 2] live_osds:  [0, 3, 5, 4]
2014-08-21T09:11:33.987 INFO:tasks.thrashosds.thrasher:choose_action: min_in 3 min_out 0 min_live 2 min_dead 0
2014-08-21T09:11:33.987 INFO:tasks.thrashosds.thrasher:fixing pg num pool .users.uid
2014-08-21T09:11:33.988 INFO:teuthology.orchestra.run.vpm064:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph pg dump --format=json'
2014-08-21T09:11:34.203 INFO:teuthology.orchestra.run.vpm064.stderr:dumped all in format json
2014-08-21T09:11:34.237 INFO:teuthology.orchestra.run.vpm064:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph osd pool set .users.uid pgp_num 18'
2014-08-21T09:11:35.373 INFO:teuthology.orchestra.run.vpm064.stderr:set pool 7 pgp_num to 18
2014-08-21T09:11:40.381 INFO:tasks.thrashosds.thrasher:in_osds:  [0, 2, 4, 5, 3]  out_osds:  [1] dead_osds:  [1, 2] live_osds:  [0, 3, 5, 4]
2014-08-21T09:11:40.382 INFO:tasks.thrashosds.thrasher:choose_action: min_in 3 min_out 0 min_live 2 min_dead 0
2014-08-21T09:11:40.382 INFO:tasks.thrashosds.thrasher:inject_pause on 5
2014-08-21T09:11:40.382 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3
2014-08-21T09:11:40.382 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False
2014-08-21T09:11:40.382 INFO:teuthology.orchestra.run.vpm063:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3'
2014-08-21T09:11:42.527 INFO:teuthology.orchestra.run.vpm063.stderr:testGetRequest (test.functional.tests.TestAccountNoContainers) ... FAIL

it does not see there was wait time as expected in config.yaml.

  rgw:
    default_idle_timeout: 1200