Ceph » CephFS

Run: http://pulpito.ceph.com/teuthology-2015-08-18_09:06:57-powercycle-hammer-testing-basic-multi/
Job: 1020696
Logs: http://qa-proxy.ceph.com/teuthology/teuthology-2015-08-18_09:06:57-powercycle-hammer-testing-basic-multi/1020696/teuthology.log

2015-08-18T18:21:12.764 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0...
2015-08-18T18:21:12.764 INFO:tasks.workunit:Running workunit suites/fsstress.sh...
2015-08-18T18:21:12.764 INFO:teuthology.orchestra.run.plana57:Running (workunit test suites/fsstress.sh): 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=697101e4dfd9822050ce401b5f6212bfd81fea89 TESTDIR="/home/ubuntu/cephtest" CEPH_ID="0" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/workunit.client.0/suites/fsstress.sh'
2015-08-18T18:21:14.264 INFO:teuthology.orchestra.run.plana57.stderr:reweighted osd.2 to 0.977567 (8640652)
2015-08-18T18:21:15.919 INFO:tasks.workunit.client.0.plana57.stdout:cc  -DNO_XFS -I/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib  fsstress.c   -o fsstress
2015-08-18T18:21:16.333 INFO:tasks.workunit.client.0.plana57.stdout:‘/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress’ -> ‘/usr/lib/ltp/testcases/bin/fsstress’
2015-08-18T18:21:16.334 INFO:tasks.workunit.client.0.plana57.stderr:cp: not writing through dangling symlink ‘/usr/lib/ltp/testcases/bin/fsstress’
2015-08-18T18:21:16.353 INFO:tasks.workunit.client.0.plana57.stderr:chmod: cannot operate on dangling symlink ‘/usr/lib/ltp/testcases/bin/fsstress’
2015-08-18T18:21:17.381 INFO:tasks.workunit.client.0.plana57.stdout:Starting fsstress /usr/lib/ltp/testcases/bin/fsstress -d fsstress-plana5711823 -l 1 -n 1000 -p 10 -v
2015-08-18T18:21:17.388 INFO:tasks.workunit.client.0.plana57.stderr:/home/ubuntu/cephtest/workunit.client.0/suites/fsstress.sh: line 24: /usr/lib/ltp/testcases/bin/fsstress: No such file or directory
2015-08-18T18:21:17.391 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0...
2015-08-18T18:21:17.391 INFO:teuthology.orchestra.run.plana57:Running: 'rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/workunit.client.0'
2015-08-18T18:21:17.559 ERROR:teuthology.parallel:Exception in parallel execution
Traceback (most recent call last):
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 82, in __exit__
    for result in self:
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/var/lib/teuthworker/src/ceph-qa-suite_hammer/tasks/workunit.py", line 361, in _run_tests
    label="workunit test {workunit}".format(workunit=workunit)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/remote.py", line 156, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 378, in run
    r.wait()
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 114, in wait
    label=self.label)
CommandFailedError: Command failed (workunit test suites/fsstress.sh) on plana57 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=697101e4dfd9822050ce401b5f6212bfd81fea89 TESTDIR="/home/ubuntu/cephtest" CEPH_ID="0" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/workunit.client.0/suites/fsstress.sh'
2015-08-18T18:21:17.674 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
  File "/home/teuthworker/src/teuthology_master/teuthology/run_tasks.py", line 53, in run_tasks
    manager = run_one_task(taskname, ctx=ctx, config=config)
  File "/home/teuthworker/src/teuthology_master/teuthology/run_tasks.py", line 41, in run_one_task
    return fn(**kwargs)
  File "/var/lib/teuthworker/src/ceph-qa-suite_hammer/tasks/workunit.py", line 115, in task
    config.get('subdir'), timeout=timeout)
  File "/var/lib/teuthworker/src/ceph-qa-suite_hammer/tasks/workunit.py", line 252, in _spawn_on_all_clients
    timeout=timeout)
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 82, in __exit__
    for result in self:
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/var/lib/teuthworker/src/ceph-qa-suite_hammer/tasks/workunit.py", line 361, in _run_tests
    label="workunit test {workunit}".format(workunit=workunit)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/remote.py", line 156, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 378, in run
    r.wait()
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 114, in wait
    label=self.label)
CommandFailedError: Command failed (workunit test suites/fsstress.sh) on plana57 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=697101e4dfd9822050ce401b5f6212bfd81fea89 TESTDIR="/home/ubuntu/cephtest" CEPH_ID="0" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/workunit.client.0/suites/fsstress.sh'
2015-08-18T18:21:17.760 ERROR:teuthology.run_tasks: Sentry event: http://sentry.ceph.com/sepia/teuthology/search?q=026e9ee477c0487cafd0b639d02e31a7
CommandFailedError: Command failed (workunit test suites/fsstress.sh) on plana57 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=697101e4dfd9822050ce401b5f6212bfd81fea89 TESTDIR="/home/ubuntu/cephtest" CEPH_ID="0" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/workunit.client.0/suites/fsstress.sh'