Project

General

Profile

Actions
Copy link

Bug #5894

closed

test_daemon teuthology failure

Added by Samuel Just over 10 years ago. Updated over 10 years ago.

Status:
Can't reproduce
Priority:
Urgent
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Development
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2013-08-06T02:37:04.714 INFO:teuthology.task.workunit.client.0.out:[10.214.132.22]: crush_ruleset: 2
2013-08-06T02:37:04.714 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + ceph osd thrash 10
2013-08-06T02:37:04.971 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: will thrash map for 10 epochs
2013-08-06T02:37:04.983 INFO:teuthology.task.workunit.client.0.out:[10.214.132.22]: OK
2013-08-06T02:37:04.983 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + echo OK
2013-08-06T02:37:04.984 DEBUG:teuthology.orchestra.run:Running [10.214.132.22]: 'sudo rm rf - /home/ubuntu/cephtest/96941/mnt.0/client.0/tmp'
2013-08-06T02:37:04.997 INFO:teuthology.task.workunit:Running workunit cephtool/test_daemon.sh...
2013-08-06T02:37:04.997 DEBUG:teuthology.orchestra.run:Running [10.214.132.22]: 'mkdir p - /home/ubuntu/cephtest/96941/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/96941/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e5d9ac64df6569c801f644c222547b7ee623e15f TESTDIR="/home/ubuntu/cephtest/96941" CEPH_ID="0" PYTHONPATH="$PYTHONPATH:/home/ubuntu/cephtest/96941/binary/usr/local/lib/python2.7/dist-packages:/home/ubuntu/cephtest/96941/binary/usr/local/lib/python2.6/dist-packages" /home/ubuntu/cephtest/96941/adjust-ulimits ceph-coverage /home/ubuntu/cephtest/96941/archive/coverage /home/ubuntu/cephtest/96941/workunit.client.0/cephtool/test_daemon.sh'
2013-08-06T02:37:05.072 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + set e
2013-08-06T02:37:05.072 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + echo note: assuming mon.a is on the currenet host
2013-08-06T02:37:05.073 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + sudo ceph daemon mon.a version
2013-08-06T02:37:05.073 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + grep version
2013-08-06T02:37:05.073 INFO:teuthology.task.workunit.client.0.out:[10.214.132.22]: note: assuming mon.a is on the currenet host
2013-08-06T02:37:05.180 INFO:teuthology.task.workunit.client.0.out:[10.214.132.22]: {"version":"0.67-rc3-30-ge5d9ac6"}
2013-08-06T02:37:05.182 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + sudo ceph daemon mon.a config get debug_ms
2013-08-06T02:37:05.182 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + grep debug_ms
2013-08-06T02:37:05.182 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + sed -e 's/.*: //' -e 's/["\}\\]//g'
2013-08-06T02:37:05.290 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + old_ms=1/1
2013-08-06T02:37:05.291 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + sudo ceph daemon mon.a config set debug_ms 13
2013-08-06T02:37:05.393 INFO:teuthology.task.workunit.client.0.out:[10.214.132.22]: { "success": ""}
2013-08-06T02:37:05.401 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + sudo ceph daemon mon.a config get debug_ms
2013-08-06T02:37:05.401 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + grep debug_ms
2013-08-06T02:37:05.402 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + sed -e 's/.*: //' -e 's/["\}\\]//g'
2013-08-06T02:37:05.511 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + new_ms=13/13
2013-08-06T02:37:05.511 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + '[' 13/13 = 13/13 ']'
2013-08-06T02:37:05.511 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + sudo ceph daemon mon.a config set debug_ms 1/1
2013-08-06T02:37:05.614 INFO:teuthology.task.workunit.client.0.out:[10.214.132.22]: { "success": ""}
2013-08-06T02:37:05.622 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + sudo ceph daemon mon.a config get debug_ms
2013-08-06T02:37:05.622 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + grep debug_ms
2013-08-06T02:37:05.622 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: ++ sed -e 's/.*: //' -e 's/["\}\\]//g'
2013-08-06T02:37:05.761 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + new_ms=1/1
2013-08-06T02:37:05.761 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + '[' 1/1 = 1/1 ']'
2013-08-06T02:37:05.762 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + expect_false sudo ceph daemon mon.a bogus_command_blah foo
2013-08-06T02:37:05.762 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + set -x
2013-08-06T02:37:05.762 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + sudo ceph daemon mon.a bogus_command_blah foo
2013-08-06T02:37:05.867 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: no valid command found; 10 closest matches:
2013-08-06T02:37:05.868 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: config set <var> <val> [<val>...]
2013-08-06T02:37:05.868 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: sync_force --yes-i-really-mean-it
2013-08-06T02:37:05.868 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: quorum_status
2013-08-06T02:37:05.868 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: git_version
2013-08-06T02:37:05.868 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: help
2013-08-06T02:37:05.869 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: config show
2013-08-06T02:37:05.869 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: get_command_descriptions
2013-08-06T02:37:05.869 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: config get <var>
2013-08-06T02:37:05.869 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: perfcounters_dump
2013-08-06T02:37:05.869 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: 2
2013-08-06T02:37:05.870 INFO:teuthology.task.workunit.client.0.out:[10.214.132.22]: -22
2013-08-06T02:37:05.874 INFO:teuthology.task.workunit.client.0.err:[10.214.132.22]: + return 1
2013-08-06T02:37:05.876 INFO:teuthology.task.workunit:Stopping cephtool on client.0...
2013-08-06T02:37:05.876 DEBUG:teuthology.orchestra.run:Running [10.214.132.22]: 'rm -rf - /home/ubuntu/cephtest/96941/workunits.list /home/ubuntu/cephtest/96941/workunit.client.0'
2013-08-06T02:37:05.953 ERROR:teuthology.run_tasks:Saw exception from tasks
Traceback (most recent call last):
File "/home/teuthworker/teuthology-next/teuthology/run_tasks.py", line 25, in run_tasks
manager = run_one_task(taskname, ctx=ctx, config=config)
File "/home/teuthworker/teuthology-next/teuthology/run_tasks.py", line 14, in run_one_task
return fn(**kwargs)
File "/home/teuthworker/teuthology-next/teuthology/task/workunit.py", line 94, in task
_spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), config.get('subdir'))
File "/home/teuthworker/teuthology-next/teuthology/task/workunit.py", line 213, in _spawn_on_all_clients
p.spawn(_run_tests, ctx, refspec, role, [unit], env, subdir)
File "/home/teuthworker/teuthology-next/teuthology/parallel.py", line 83, in exit
for result in self:
File "/home/teuthworker/teuthology-next/teuthology/parallel.py", line 100, in next
resurrect_traceback(result)
File "/home/teuthworker/teuthology-next/teuthology/parallel.py", line 19, in capture_traceback
return func(*args, **kwargs)
File "/home/teuthworker/teuthology-next/teuthology/task/workunit.py", line 301, in _run_tests
args=args,
File "/home/teuthworker/teuthology-next/teuthology/orchestra/remote.py", line 43, in run
r = self._runner(client=self.ssh, **kwargs)
File "/home/teuthworker/teuthology-next/teuthology/orchestra/run.py", line 267, in run
r.exitstatus = _check_status(r.exitstatus)
File "/home/teuthworker/teuthology-next/teuthology/orchestra/run.py", line 263, in _check_status
raise CommandFailedError(command=r.command, exitstatus=status, node=host)
CommandFailedError: Command failed on 10.214.132.22 with status 1: 'mkdir p - /home/ubuntu/cephtest/96941/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/96941/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e5d9ac64df6569c801f644c222547b7ee623e15f TESTDIR="/home/ubuntu/cephtest/96941" CEPH_ID="0" PYTHONPATH="$PYTHONPATH:/home/ubuntu/cephtest/96941/binary/usr/local/lib/python2.7/dist-packages:/home/ubuntu/cephtest/96941/binary/usr/local/lib/python2.6/dist-packages" /home/ubuntu/cephtest/96941/adjust-ulimits ceph-coverage /home/ubuntu/cephtest/96941/archive/coverage /home/ubuntu/cephtest/96941/workunit.client.0/cephtool/test_daemon.sh'
2013-08-06T02:37:06.011 DEBUG:teuthology.run_tasks:Unwinding manager <contextlib.GeneratorContextManager object at 0x32d03d0>
2013-08-06T02:37:06.011 ERROR:teuthology.contextutil:Saw exception from nested tasks

Actions
Copy link
 #1

Updated by Samuel Just over 10 years ago

ubuntu@teuthology:/a/teuthology-2013-08-05_01:00:16-rados-next-testing-basic-plana/96941

Actions
Copy link
 #2

Updated by Dan Mick over 10 years ago

Yeah, this was fixed with 9466a0b152beeab894be66c6528495ca01a1974a I believe. If it happens again with a version >= that, let me know.

Actions
Copy link
 #3

Updated by Dan Mick over 10 years ago

  • Status changed from New to Can't reproduce
  • Source changed from other to Development
Actions
Copy link

Also available in: Atom PDF