Project

General

Profile

Actions
Copy link

Bug #38766

closed

mgr/dashboard: QA run failure "ERROR: test_selftest_run (tasks.mgr.test_module_selftest.TestModuleSelftest)"

Added by Laura Paduano about 5 years ago. Updated about 3 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Volker Theile
Category:
Testing & QA
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
26986
Crash signature (v1):
Crash signature (v2):

Description

suite: rados/mgr

2019-03-15T11:28:32.554 INFO:tasks.cephfs_test_runner:ERROR: test_selftest_run (tasks.mgr.test_module_selftest.TestModuleSelftest)
2019-03-15T11:28:32.554 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2019-03-15T11:28:32.554 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2019-03-15T11:28:32.554 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-lpaduano-testing-26914/qa/tasks/mgr/test_module_selftest.py", line 73, in test_selftest_run
2019-03-15T11:28:32.554 INFO:tasks.cephfs_test_runner:    self.mgr_cluster.mon_manager.raw_cluster_cmd("mgr", "self-test", "run")
2019-03-15T11:28:32.554 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-lpaduano-testing-26914/qa/tasks/ceph_manager.py", line 1157, in raw_cluster_cmd
2019-03-15T11:28:32.554 INFO:tasks.cephfs_test_runner:    stdout=StringIO(),
2019-03-15T11:28:32.554 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 193, in run
2019-03-15T11:28:32.555 INFO:tasks.cephfs_test_runner:    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
2019-03-15T11:28:32.555 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 435, in run
2019-03-15T11:28:32.555 INFO:tasks.cephfs_test_runner:    r.wait()
2019-03-15T11:28:32.555 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 162, in wait
2019-03-15T11:28:32.555 INFO:tasks.cephfs_test_runner:    self._raise_for_status()
2019-03-15T11:28:32.555 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 184, in _raise_for_status
2019-03-15T11:28:32.555 INFO:tasks.cephfs_test_runner:    node=self.hostname, label=self.label
2019-03-15T11:28:32.555 INFO:tasks.cephfs_test_runner:CommandFailedError: Command failed on mira117 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr self-test run'
2019-03-15T11:28:32.555 INFO:tasks.cephfs_test_runner:
2019-03-15T11:28:32.556 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2019-03-15T11:28:32.556 INFO:tasks.cephfs_test_runner:Ran 15 tests in 1551.395s
2019-03-15T11:28:32.556 INFO:tasks.cephfs_test_runner:
2019-03-15T11:28:32.556 INFO:tasks.cephfs_test_runner:FAILED (errors=1)
2019-03-15T11:28:32.556 INFO:tasks.cephfs_test_runner:
2019-03-15T11:28:32.556 INFO:tasks.cephfs_test_runner:======================================================================
2019-03-15T11:28:32.556 INFO:tasks.cephfs_test_runner:ERROR: test_selftest_run (tasks.mgr.test_module_selftest.TestModuleSelftest)
2019-03-15T11:28:32.557 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2019-03-15T11:28:32.557 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2019-03-15T11:28:32.557 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-lpaduano-testing-26914/qa/tasks/mgr/test_module_selftest.py", line 73, in test_selftest_run
2019-03-15T11:28:32.557 INFO:tasks.cephfs_test_runner:    self.mgr_cluster.mon_manager.raw_cluster_cmd("mgr", "self-test", "run")
2019-03-15T11:28:32.557 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-lpaduano-testing-26914/qa/tasks/ceph_manager.py", line 1157, in raw_cluster_cmd
2019-03-15T11:28:32.557 INFO:tasks.cephfs_test_runner:    stdout=StringIO(),
2019-03-15T11:28:32.557 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 193, in run
2019-03-15T11:28:32.557 INFO:tasks.cephfs_test_runner:    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
2019-03-15T11:28:32.557 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 435, in run
2019-03-15T11:28:32.558 INFO:tasks.cephfs_test_runner:    r.wait()
2019-03-15T11:28:32.558 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 162, in wait
2019-03-15T11:28:32.558 INFO:tasks.cephfs_test_runner:    self._raise_for_status()
2019-03-15T11:28:32.558 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 184, in _raise_for_status
2019-03-15T11:28:32.558 INFO:tasks.cephfs_test_runner:    node=self.hostname, label=self.label
2019-03-15T11:28:32.558 INFO:tasks.cephfs_test_runner:CommandFailedError: Command failed on mira117 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr self-test run'
2019-03-15T11:28:32.558 INFO:tasks.cephfs_test_runner:
2019-03-15T11:28:32.558 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 89, in run_tasks
    manager.__enter__()
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-lpaduano-testing-26914/qa/tasks/cephfs_test_runner.py", line 207, in task
    raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests)))
RuntimeError: Test failure: test_selftest_run (tasks.mgr.test_module_selftest.TestModuleSelftest)

Also see:
http://pulpito.ceph.com/pnawracay-2019-03-15_10:35:52-rados:mgr-wip-lpaduano-testing-26914-distro-basic-mira/
http://qa-proxy.ceph.com/teuthology/pnawracay-2019-03-15_10:35:52-rados:mgr-wip-lpaduano-testing-26914-distro-basic-mira/3728438/teuthology.log

Actions
Copy link
 #1

Updated by Sebastian Wagner about 5 years ago

the real error in the qa log was a bit earlier:

2019-03-15T11:28:31.324 INFO:tasks.ceph.mgr.x.mira117.stderr:2019-03-15 11:28:31.312 7f4973377700 -1 mgr.server reply reply (22) Invalid argument Traceback (most recent call last):
2019-03-15T11:28:31.324 INFO:tasks.ceph.mgr.x.mira117.stderr:  File "/usr/share/ceph/mgr/mgr_module.py", line 863, in _handle_command
2019-03-15T11:28:31.324 INFO:tasks.ceph.mgr.x.mira117.stderr:    return self.handle_command(inbuf, cmd)
2019-03-15T11:28:31.324 INFO:tasks.ceph.mgr.x.mira117.stderr:  File "/usr/share/ceph/mgr/selftest/module.py", line 114, in handle_command
2019-03-15T11:28:31.324 INFO:tasks.ceph.mgr.x.mira117.stderr:    self._self_test()
2019-03-15T11:28:31.325 INFO:tasks.ceph.mgr.x.mira117.stderr:  File "/usr/share/ceph/mgr/selftest/module.py", line 213, in _self_test
2019-03-15T11:28:31.325 INFO:tasks.ceph.mgr.x.mira117.stderr:    self._self_test_config()
2019-03-15T11:28:31.325 INFO:tasks.ceph.mgr.x.mira117.stderr:  File "/usr/share/ceph/mgr/selftest/module.py", line 313, in _self_test_config
2019-03-15T11:28:31.325 INFO:tasks.ceph.mgr.x.mira117.stderr:    assert self.get_module_option_ex("foo", "bar") is None
2019-03-15T11:28:31.325 INFO:tasks.ceph.mgr.x.mira117.stderr:  File "/usr/share/ceph/mgr/mgr_module.py", line 942, in get_module_option_ex
2019-03-15T11:28:31.325 INFO:tasks.ceph.mgr.x.mira117.stderr:    r = self._ceph_get_module_option(module, key)
2019-03-15T11:28:31.325 INFO:tasks.ceph.mgr.x.mira117.stderr:TypeError: ceph_get_module_option() takes exactly 3 arguments (2 given)
2019-03-15T11:28:31.326 INFO:tasks.ceph.mgr.x.mira117.stderr:
2019-03-15T11:28:31.326 INFO:teuthology.orchestra.run.mira117.stderr:2019-03-15 11:28:31.313 7f4f42ffd700  1 -- 172.21.9.124:0/3472646312 <== mgr.8596 v2:172.21.9.124:6800/111620 1 ==== command_reply(tid 0: -22 Traceback (most recent call last):
2019-03-15T11:28:31.326 INFO:teuthology.orchestra.run.mira117.stderr:  File "/usr/share/ceph/mgr/mgr_module.py", line 863, in _handle_command
2019-03-15T11:28:31.326 INFO:teuthology.orchestra.run.mira117.stderr:    return self.handle_command(inbuf, cmd)
2019-03-15T11:28:31.326 INFO:teuthology.orchestra.run.mira117.stderr:  File "/usr/share/ceph/mgr/selftest/module.py", line 114, in handle_command
2019-03-15T11:28:31.327 INFO:teuthology.orchestra.run.mira117.stderr:    self._self_test()
2019-03-15T11:28:31.327 INFO:teuthology.orchestra.run.mira117.stderr:  File "/usr/share/ceph/mgr/selftest/module.py", line 213, in _self_test
2019-03-15T11:28:31.327 INFO:teuthology.orchestra.run.mira117.stderr:    self._self_test_config()
2019-03-15T11:28:31.327 INFO:teuthology.orchestra.run.mira117.stderr:  File "/usr/share/ceph/mgr/selftest/module.py", line 313, in _self_test_config
2019-03-15T11:28:31.327 INFO:teuthology.orchestra.run.mira117.stderr:    assert self.get_module_option_ex("foo", "bar") is None
2019-03-15T11:28:31.327 INFO:teuthology.orchestra.run.mira117.stderr:  File "/usr/share/ceph/mgr/mgr_module.py", line 942, in get_module_option_ex
2019-03-15T11:28:31.327 INFO:teuthology.orchestra.run.mira117.stderr:    r = self._ceph_get_module_option(module, key)
2019-03-15T11:28:31.327 INFO:teuthology.orchestra.run.mira117.stderr:TypeError: ceph_get_module_option() takes exactly 3 arguments (2 given)
2019-03-15T11:28:31.328 INFO:teuthology.orchestra.run.mira117.stderr:) v1 ==== 699+0+0 (crc 0 0 0) 0x7f4f4c000e70 con 0x7f4f30020b90
2019-03-15T11:28:31.328 INFO:teuthology.orchestra.run.mira117.stderr:2019-03-15 11:28:31.313 7f4f42ffd700 20 mgrc handle_command_reply command_reply(tid 0: -22 Traceback (most recent call last):
2019-03-15T11:28:31.328 INFO:teuthology.orchestra.run.mira117.stderr:  File "/usr/share/ceph/mgr/mgr_module.py", line 863, in _handle_command
2019-03-15T11:28:31.328 INFO:teuthology.orchestra.run.mira117.stderr:    return self.handle_command(inbuf, cmd)
2019-03-15T11:28:31.328 INFO:teuthology.orchestra.run.mira117.stderr:  File "/usr/share/ceph/mgr/selftest/module.py", line 114, in handle_command
2019-03-15T11:28:31.328 INFO:teuthology.orchestra.run.mira117.stderr:    self._self_test()
2019-03-15T11:28:31.328 INFO:teuthology.orchestra.run.mira117.stderr:  File "/usr/share/ceph/mgr/selftest/module.py", line 213, in _self_test
2019-03-15T11:28:31.329 INFO:teuthology.orchestra.run.mira117.stderr:    self._self_test_config()
2019-03-15T11:28:31.329 INFO:teuthology.orchestra.run.mira117.stderr:  File "/usr/share/ceph/mgr/selftest/module.py", line 313, in _self_test_config
2019-03-15T11:28:31.329 INFO:teuthology.orchestra.run.mira117.stderr:    assert self.get_module_option_ex("foo", "bar") is None
2019-03-15T11:28:31.329 INFO:teuthology.orchestra.run.mira117.stderr:  File "/usr/share/ceph/mgr/mgr_module.py", line 942, in get_module_option_ex
2019-03-15T11:28:31.329 INFO:teuthology.orchestra.run.mira117.stderr:    r = self._ceph_get_module_option(module, key)
2019-03-15T11:28:31.329 INFO:teuthology.orchestra.run.mira117.stderr:TypeError: ceph_get_module_option() takes exactly 3 arguments (2 given)
2019-03-15T11:28:31.329 INFO:teuthology.orchestra.run.mira117.stderr:) v1
2019-03-15T11:28:31.339 INFO:teuthology.orchestra.run.mira117.stderr:Error EINVAL: Traceback (most recent call last):
2019-03-15T11:28:31.339 INFO:teuthology.orchestra.run.mira117.stderr:  File "/usr/share/ceph/mgr/mgr_module.py", line 863, in _handle_command
2019-03-15T11:28:31.339 INFO:teuthology.orchestra.run.mira117.stderr:    return self.handle_command(inbuf, cmd)
2019-03-15T11:28:31.339 INFO:teuthology.orchestra.run.mira117.stderr:  File "/usr/share/ceph/mgr/selftest/module.py", line 114, in handle_command
2019-03-15T11:28:31.339 INFO:teuthology.orchestra.run.mira117.stderr:    self._self_test()
2019-03-15T11:28:31.339 INFO:teuthology.orchestra.run.mira117.stderr:  File "/usr/share/ceph/mgr/selftest/module.py", line 213, in _self_test
2019-03-15T11:28:31.340 INFO:teuthology.orchestra.run.mira117.stderr:    self._self_test_config()
2019-03-15T11:28:31.340 INFO:teuthology.orchestra.run.mira117.stderr:  File "/usr/share/ceph/mgr/selftest/module.py", line 313, in _self_test_config
2019-03-15T11:28:31.340 INFO:teuthology.orchestra.run.mira117.stderr:    assert self.get_module_option_ex("foo", "bar") is None
2019-03-15T11:28:31.340 INFO:teuthology.orchestra.run.mira117.stderr:  File "/usr/share/ceph/mgr/mgr_module.py", line 942, in get_module_option_ex
2019-03-15T11:28:31.340 INFO:teuthology.orchestra.run.mira117.stderr:    r = self._ceph_get_module_option(module, key)
2019-03-15T11:28:31.340 INFO:teuthology.orchestra.run.mira117.stderr:TypeError: ceph_get_module_option() takes exactly 3 arguments (2 given)
2019-03-15T11:28:31.340 INFO:teuthology.orchestra.run.mira117.stderr:
Actions
Copy link
 #2

Updated by Sebastian Wagner about 5 years ago

this would be another case, where mypy could have saved some pain.

Actions
Copy link
 #3

Updated by Volker Theile about 5 years ago

  • Status changed from New to In Progress
  • Assignee set to Volker Theile
Actions
Copy link
 #4

Updated by Ricardo Marques about 5 years ago

  • Priority changed from Normal to Urgent
Actions
Copy link
 #5

Updated by Volker Theile about 5 years ago

  • Pull request ID set to 26986
Actions
Copy link
 #6

Updated by Volker Theile about 5 years ago

  • Status changed from In Progress to Fix Under Review
Actions
Copy link
 #7

Updated by Volker Theile about 5 years ago

  • Status changed from Fix Under Review to Resolved
Actions
Copy link
 #8

Updated by Ernesto Puerta about 3 years ago

  • Project changed from mgr to Dashboard
  • Category changed from 151 to Testing & QA
Actions
Copy link

Also available in: Atom PDF