Tools » teuthology

See http://teuthology.front.sepia.ceph.com/archive/teuthology-2013-08-23_01:30:05-upgrade-next-testing-basic-plana/1590/teuthology.log

Note how much more concise and useful the remote traceback is.

2013-08-23T03:34:21.103 INFO:teuthology.task.workunit.client.0.err:[10.214.133.32]: + ceph pg 0.0 query
2013-08-23T03:34:21.222 INFO:teuthology.task.workunit.client.0.err:[10.214.133.32]: Traceback (most recent call last):
2013-08-23T03:34:21.223 INFO:teuthology.task.workunit.client.0.err:[10.214.133.32]: File "/usr/bin/ceph", line 774, in <module>
2013-08-23T03:34:21.223 INFO:teuthology.task.workunit.client.0.err:[10.214.133.32]: sys.exit(main())
2013-08-23T03:34:21.223 INFO:teuthology.task.workunit.client.0.err:[10.214.133.32]: File "/usr/bin/ceph", line 698, in main
2013-08-23T03:34:21.223 INFO:teuthology.task.workunit.client.0.err:[10.214.133.32]: inbuf)
2013-08-23T03:34:21.224 INFO:teuthology.task.workunit.client.0.err:[10.214.133.32]: File "/usr/lib/python2.7/dist-packages/ceph_argparse.py", line 1044, in send_command
2013-08-23T03:34:21.224 INFO:teuthology.task.workunit.client.0.err:[10.214.133.32]: raise RuntimeError('"{0}": exception {1}'.format(cmd, e))
2013-08-23T03:34:21.224 INFO:teuthology.task.workunit.client.0.err:[10.214.133.32]: RuntimeError: "['pg', '0.0', 'query']": exception No JSON object could be decoded
2013-08-23T03:34:21.233 INFO:teuthology.task.workunit.client.0.err:[10.214.133.32]: + rm /tmp/test_invalid.5688
2013-08-23T03:34:21.234 INFO:teuthology.task.workunit.client.0.err:[10.214.133.32]: rm: cannot remove `/tmp/test_invalid.5688': No such file or directory
2013-08-23T03:34:21.236 INFO:teuthology.task.workunit:Stopping cephtool/test.sh on client.0...
2013-08-23T03:34:21.236 DEBUG:teuthology.orchestra.run:Running [10.214.133.32]: 'rm rf - /home/ubuntu/cephtest/1590/workunits.list /home/ubuntu/cephtest/1590/workunit.client.0'
2013-08-23T03:34:21.315 ERROR:teuthology.run_tasks:Saw exception from tasks
Traceback (most recent call last):
File "/home/teuthworker/teuthology-next/teuthology/run_tasks.py", line 25, in run_tasks
manager = run_one_task(taskname, ctx=ctx, config=config)
File "/home/teuthworker/teuthology-next/teuthology/run_tasks.py", line 14, in run_one_task
return fn(**kwargs)
File "/home/teuthworker/teuthology-next/teuthology/task/workunit.py", line 94, in task
_spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), config.get('subdir'))
File "/home/teuthworker/teuthology-next/teuthology/task/workunit.py", line 213, in _spawn_on_all_clients
p.spawn(_run_tests, ctx, refspec, role, [unit], env, subdir)
File "/home/teuthworker/teuthology-next/teuthology/parallel.py", line 83, in exit
for result in self:
File "/home/teuthworker/teuthology-next/teuthology/parallel.py", line 100, in next
resurrect_traceback(result)
File "/home/teuthworker/teuthology-next/teuthology/parallel.py", line 19, in capture_traceback
return func(*args, **kwargs)
File "/home/teuthworker/teuthology-next/teuthology/task/workunit.py", line 301, in _run_tests
args=args,
File "/home/teuthworker/teuthology-next/teuthology/orchestra/remote.py", line 47, in run
r = self._runner(client=self.ssh, **kwargs)
File "/home/teuthworker/teuthology-next/teuthology/orchestra/run.py", line 267, in run
r.exitstatus = _check_status(r.exitstatus)
File "/home/teuthworker/teuthology-next/teuthology/orchestra/run.py", line 263, in _check_status
raise CommandFailedError(command=r.command, exitstatus=status, node=host)
CommandFailedError: Command failed on 10.214.133.32 with status 1: 'mkdir p - /home/ubuntu/cephtest/1590/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/1590/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b003e5fddca70f2d390e00bd71596e784d2ada0a TESTDIR="/home/ubuntu/cephtest/1590" CEPH_ID="0" PYTHONPATH="$PYTHONPATH:/home/ubuntu/cephtest/1590/binary/usr/local/lib/python2.7/dist-packages:/home/ubuntu/cephtest/1590/binary/usr/local/lib/python2.6/dist-packages" /home/ubuntu/cephtest/1590/adjust-ulimits ceph-coverage /home/ubuntu/cephtest/1590/archive/coverage /home/ubuntu/cephtest/1590/workunit.client.0/cephtool/test.sh'