Project

General

Profile

Bug #9456

rocksdb testing with powercycling fails on trusty

Added by Tamilarasi muthamizhan over 9 years ago. Updated over 9 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

when ceph_osd_objectstore is set to rocksdb with powercycling on trusty, it fails with the following message,

2014-09-12 00:36:24,174.174 INFO:teuthology.orchestra.run.mira002:Running: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=HEAD TESTDIR="/home/ubuntu/cephtest" CEPH_ID="0" adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/workunit.client.0/suites/blogbench.sh'
2014-09-12 00:36:29,051.051 INFO:tasks.ceph.osd.2:Stopped
2014-09-12 00:36:29,054.054 INFO:tasks.thrashosds.thrasher:Testing ceph_objectstore_tool on down osd
2014-09-12 00:36:29,055.055 INFO:teuthology.orchestra.run.mira002:Running: 'sudo ceph_objectstore_tool --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op list-pgs'
2014-09-12 00:36:29,172.172 INFO:teuthology.orchestra.run.mira002.stderr:Mount failed with '(22) Invalid argument'
Traceback (most recent call last):
  File "/home/tamil/test_review/teuthology/virtualenv/local/lib/python2.7/site-packages/gevent/greenlet.py", line 390, in run
    result = self._run(*self.args, **self.kwargs)
  File "/home/tamil/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 502, in do_thrash
    self.choose_action()()
  File "/home/tamil/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 135, in kill_osd
    proc = exp_remote.run(args=cmd, wait=True, check_status=True, stdout=StringIO())
  File "/home/tamil/test_review/teuthology/teuthology/orchestra/remote.py", line 124, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/tamil/test_review/teuthology/teuthology/orchestra/run.py", line 357, in run
    r.wait()
  File "/home/tamil/test_review/teuthology/teuthology/orchestra/run.py", line 104, in wait
    exitstatus=status, node=self.hostname)
CommandFailedError: Command failed on mira002 with status 1: 'sudo ceph_objectstore_tool --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op list-pgs'
<Greenlet at 0x1e2b870: <bound method Thrasher.do_thrash of <tasks.ceph_manager.Thrasher instance at 0x1e37c20>>> failed with CommandFailedError

the config file used is shown below,

ubuntu@mira002:~/rocksdb_trusty$ cat orig.config.yaml 
overrides:
  ceph:
    conf:
      osd:
        keyvaluestore_backend: rocksdb
        osd_objectstore: keyvaluestore-dev
roles:
- - mon.a
  - mds.0
  - osd.0
  - osd.1
  - osd.2
  - client.0
targets:
  ubuntu@mira002.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYuQd2eCETPB1aNhqxBt21cj7Rw6O2HzzPm8DvLroaDxDAdbvY6oFSHv/i1uNL4/NJN5EDBgqVF3j3ITHmxggmx1bFK5zFrdLuzq5wI1/D+DICgIH5RJqEfg7qTMqY+EKIXN3Tcwnb4ceYiUc3RgWFlBK+Igm6bA/Mf/4fDyn+VWtEJZGXFPyuYIFFrXprmUXLKCv9GiQAVbN+iql/OiGTrQ0aRI8Cgu7EMr04KLA89UGV68owvKJcKzltqmaL61nnlqT7D5AO5OmEQ4XqzdWEY7cF1NmCTiTiY4pxu4Lcz0xstcREl9WnxwlzRjlI8lJuXoqWVo6HIDtNtlL5Dksp
tasks:
- chef: null
- install:
    branch: giant
- ceph: null
- thrashosds:
    chance_down: 1.0
    powercycle: true
    timeout: 600
- ceph-fuse: null
- workunit:
    clients:
      client.0:
      - suites/blogbench.sh

copied the logs to :/home/ubuntu/rocksdb_trusty

History

#1 Updated by Josh Durgin over 9 years ago

  • Status changed from New to Rejected

teuthology needs to pass --type keyvaluestore-dev to ceph_objectstore_tool when using a key value store backend.

Also available in: Atom PDF