Project

General

Profile

Bug #43230

mgr/dashboard: test_diskprediction_local failure when running the backend API tests (vstart)

Added by Laura Paduano about 1 year ago. Updated 12 months ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

Error EINVAL: Traceback (most recent call last):
  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/src/pybind/mgr/mgr_module.py", line 1062, in _handle_command
    return self.handle_command(inbuf, cmd)
  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/src/pybind/mgr/selftest/module.py", line 143, in handle_command
    r = self.remote(command['module'], "self_test")
  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/src/pybind/mgr/mgr_module.py", line 1424, in remote
    args, kwargs)
ImportError: Module not found

2019-12-10 14:56:20,822.822 INFO:__main__:test_diskprediction_local (tasks.mgr.test_module_selftest.TestModuleSelftest) ... ERROR
2019-12-10 14:56:20,823.823 INFO:__main__:Running ['./bin/ceph', 'log', 'Ended test tasks.mgr.test_module_selftest.TestModuleSelftest.test_diskprediction_local']
2019-12-10 14:56:21,859.859 INFO:__main__:Stopped test: test_diskprediction_local (tasks.mgr.test_module_selftest.TestModuleSelftest) in 34.877439s
2019-12-10 14:56:21,860.860 INFO:__main__:
2019-12-10 14:56:21,860.860 INFO:__main__:======================================================================
2019-12-10 14:56:21,860.860 INFO:__main__:ERROR: test_diskprediction_local (tasks.mgr.test_module_selftest.TestModuleSelftest)
2019-12-10 14:56:21,861.861 INFO:__main__:----------------------------------------------------------------------
2019-12-10 14:56:21,861.861 INFO:__main__:Traceback (most recent call last):
2019-12-10 14:56:21,861.861 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/test_module_selftest.py", line 51, in test_diskprediction_local
2019-12-10 14:56:21,861.861 INFO:__main__:    self._selftest_plugin("diskprediction_local")
2019-12-10 14:56:21,861.861 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/test_module_selftest.py", line 34, in _selftest_plugin
2019-12-10 14:56:21,861.861 INFO:__main__:    "mgr", "self-test", "module", module_name)
2019-12-10 14:56:21,861.861 INFO:__main__:  File "../qa/tasks/vstart_runner.py", line 937, in raw_cluster_cmd
2019-12-10 14:56:21,862.862 INFO:__main__:    proc = self.controller.run([os.path.join(BIN_PREFIX, "ceph")] + list(args), **kwargs)
2019-12-10 14:56:21,862.862 INFO:__main__:  File "../qa/tasks/vstart_runner.py", line 390, in run
2019-12-10 14:56:21,862.862 INFO:__main__:    proc.wait()
2019-12-10 14:56:21,862.862 INFO:__main__:  File "../qa/tasks/vstart_runner.py", line 190, in wait
2019-12-10 14:56:21,862.862 INFO:__main__:    raise CommandFailedError(self.args, self.exitstatus)
2019-12-10 14:56:21,862.862 INFO:__main__:CommandFailedError: Command failed with status 22: ['./bin/ceph', 'mgr', 'self-test', 'module', 'diskprediction_local']
2019-12-10 14:56:21,862.862 INFO:__main__:
2019-12-10 14:56:21,863.863 INFO:__main__:----------------------------------------------------------------------
2019-12-10 14:56:21,863.863 INFO:__main__:Ran 229 tests in 2594.936s
2019-12-10 14:56:21,863.863 INFO:__main__:
2019-12-10 14:56:21,863.863 INFO:__main__:FAILED (errors=1)
2019-12-10 14:56:21,863.863 INFO:__main__:
2019-12-10 14:56:21,863.863 INFO:__main__:======================================================================
2019-12-10 14:56:21,864.864 INFO:__main__:ERROR: test_diskprediction_local (tasks.mgr.test_module_selftest.TestModuleSelftest)
2019-12-10 14:56:21,864.864 INFO:__main__:----------------------------------------------------------------------
2019-12-10 14:56:21,864.864 INFO:__main__:Traceback (most recent call last):
2019-12-10 14:56:21,864.864 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/test_module_selftest.py", line 51, in test_diskprediction_local
2019-12-10 14:56:21,864.864 INFO:__main__:    self._selftest_plugin("diskprediction_local")
2019-12-10 14:56:21,864.864 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/test_module_selftest.py", line 34, in _selftest_plugin
2019-12-10 14:56:21,864.864 INFO:__main__:    "mgr", "self-test", "module", module_name)
2019-12-10 14:56:21,865.865 INFO:__main__:  File "../qa/tasks/vstart_runner.py", line 937, in raw_cluster_cmd
2019-12-10 14:56:21,865.865 INFO:__main__:    proc = self.controller.run([os.path.join(BIN_PREFIX, "ceph")] + list(args), **kwargs)
2019-12-10 14:56:21,865.865 INFO:__main__:  File "../qa/tasks/vstart_runner.py", line 390, in run
2019-12-10 14:56:21,865.865 INFO:__main__:    proc.wait()
2019-12-10 14:56:21,865.865 INFO:__main__:  File "../qa/tasks/vstart_runner.py", line 190, in wait
2019-12-10 14:56:21,865.865 INFO:__main__:    raise CommandFailedError(self.args, self.exitstatus)
2019-12-10 14:56:21,865.865 INFO:__main__:CommandFailedError: Command failed with status 22: ['./bin/ceph', 'mgr', 'self-test', 'module', 'diskprediction_local']

- https://jenkins.ceph.com/job/ceph-dashboard-pr-backend/511/consoleFull
- also reproducible on my local environment

049c32d0-1b5f-11ea-8274-832bf91448a4.png View (52.6 KB) Volker Theile, 12/10/2019 03:24 PM


Related issues

Related to mgr - Bug #42764: Test failure: test_diskprediction_local (tasks.mgr.test_module_selftest.TestModuleSelftest) Resolved

History

#1 Updated by Laura Paduano about 1 year ago

  • Related to Bug #42764: Test failure: test_diskprediction_local (tasks.mgr.test_module_selftest.TestModuleSelftest) added

#2 Updated by Laura Paduano about 1 year ago

Volker mentioned it could be related to https://github.com/ceph/ceph/pull/32102 ?

#3 Updated by Volker Theile about 1 year ago

IMO https://github.com/ceph/ceph/pull/32102/files#diff-71670949e7d2a0a0d442bdad2b1e2debR18 is the reason for this. Importing 'scipy' fails and causes E2E to fail:

   verifies editing functionality for manager modules
      ✓ should test editing on ansible module (4 secs)
      ✓ should test editing on deepsea module (6 secs)
      ✗ should test editing on diskprediction_local module (0.729 sec)
        - Failed: element click intercepted: Element <button _ngcontent-yae-c15="" type="button" tabindex="0" class="btn btn-secondary disabled">...</button> is not clickable at point (66, 165). Other element would receive the click: <div _ngcontent-yae-c15="" class="btn-group" dropdown="">...</div>
          (Session info: headless chrome=78.0.3904.108)
          (Driver info: chromedriver=78.0.3904.70 (edb9c9f3de0247fd912a77b7f6cae7447f6d3ad5-refs/branch-heads/3904@{#800}),platform=Linux 5.3.0-24-generic x86_64)
          (Session info: headless chrome=78.0.3904.108)
          (Driver info: chromedriver=78.0.3904.70 (edb9c9f3de0247fd912a77b7f6cae7447f6d3ad5-refs/branch-heads/3904@{#800}),platform=Linux 5.3.0-24-generic x86_64)
            at Object.checkLegacyResponse (/ceph/src/pybind/mgr/dashboard/frontend/node_modules/selenium-webdriver/lib/error.js:546:15)
            at parseHttpResponse (/ceph/src/pybind/mgr/dashboard/frontend/node_modules/selenium-webdriver/lib/http.js:509:13)
            at doSend.then.response (/ceph/src/pybind/mgr/dashboard/frontend/node_modules/selenium-webdriver/lib/http.js:441:30)
            at process._tickCallback (internal/process/next_tick.js:68:7)Error: 
            at ElementArrayFinder.applyAction_ (/ceph/src/pybind/mgr/dashboard/frontend/node_modules/protractor/built/element.js:459:27)
            at ElementArrayFinder.(anonymous function).args [as click] (/ceph/src/pybind/mgr/dashboard/frontend/node_modules/protractor/built/element.js:91:29)
            at ElementFinder.(anonymous function).args [as click] (/ceph/src/pybind/mgr/dashboard/frontend/node_modules/protractor/built/element.js:831:22)
            at ManagerModulesPageHelper.<anonymous> (/ceph/src/pybind/mgr/dashboard/frontend/e2e/cluster/mgr-modules.po.ts:20:59)
            at step (/ceph/src/pybind/mgr/dashboard/frontend/node_modules/tslib/tslib.js:136:27)
            at Object.next (/ceph/src/pybind/mgr/dashboard/frontend/node_modules/tslib/tslib.js:117:57)
            at fulfilled (/ceph/src/pybind/mgr/dashboard/frontend/node_modules/tslib/tslib.js:107:62)
            at process._tickCallback (internal/process/next_tick.js:68:7)
        From asynchronous test: 
        Error: 

Reason: The details table is empty.

#4 Updated by Laura Paduano about 1 year ago

  • Pull request ID set to 32151

#5 Updated by Laura Paduano about 1 year ago

For the e2e Jenkins system see: https://github.com/ceph/ceph-build/pull/1448

#6 Updated by Kefu Chai about 1 year ago

  • Status changed from New to Resolved

#7 Updated by Laura Paduano about 1 year ago

  • Category deleted (dashboard/qa)
  • Status changed from Resolved to New
  • Source deleted (Q/A)
  • Severity deleted (3 - minor)

We're still facing issues and maybe python-scipy needs to be added to ceph.spec.in as suggested by Kefu?
Or is this related to the machines (setup) that were used for the Job? Because two builds were successful yesterday, today it started to fail again.

2019-12-13 08:57:45,320.320 INFO:tasks.mgr.mgr_test_case:Restarted after module load (new active y/20096)
2019-12-13 08:57:45,323.323 INFO:__main__:Running ['./bin/ceph', 'mgr', 'self-test', 'module', 'diskprediction_local']
2019-12-13T08:57:45.503+0000 7efd5b927700 -1 WARNING: all dangerous and experimental features are enabled.
2019-12-13T08:57:45.568+0000 7efd5b927700 -1 WARNING: all dangerous and experimental features are enabled.
Error EINVAL: Traceback (most recent call last):
  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/src/pybind/mgr/mgr_module.py", line 1062, in _handle_command
    return self.handle_command(inbuf, cmd)
  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/src/pybind/mgr/selftest/module.py", line 143, in handle_command
    r = self.remote(command['module'], "self_test")
  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/src/pybind/mgr/mgr_module.py", line 1424, in remote
    args, kwargs)
ImportError: Module not found

2019-12-13 08:57:45,930.930 INFO:__main__:test_diskprediction_local (tasks.mgr.test_module_selftest.TestModuleSelftest) ... ERROR
2019-12-13 08:57:45,930.930 INFO:__main__:Running ['./bin/ceph', 'log', 'Ended test tasks.mgr.test_module_selftest.TestModuleSelftest.test_diskprediction_local']
2019-12-13 08:57:46,897.897 INFO:__main__:Stopped test: test_diskprediction_local (tasks.mgr.test_module_selftest.TestModuleSelftest) in 34.837163s
2019-12-13 08:57:46,898.898 INFO:__main__:
2019-12-13 08:57:46,898.898 INFO:__main__:======================================================================
2019-12-13 08:57:46,899.899 INFO:__main__:ERROR: test_diskprediction_local (tasks.mgr.test_module_selftest.TestModuleSelftest)
2019-12-13 08:57:46,899.899 INFO:__main__:----------------------------------------------------------------------
2019-12-13 08:57:46,899.899 INFO:__main__:Traceback (most recent call last):
2019-12-13 08:57:46,900.900 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/test_module_selftest.py", line 51, in test_diskprediction_local
2019-12-13 08:57:46,900.900 INFO:__main__:    self._selftest_plugin("diskprediction_local")
2019-12-13 08:57:46,900.900 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/test_module_selftest.py", line 34, in _selftest_plugin
2019-12-13 08:57:46,900.900 INFO:__main__:    "mgr", "self-test", "module", module_name)
2019-12-13 08:57:46,900.900 INFO:__main__:  File "../qa/tasks/vstart_runner.py", line 936, in raw_cluster_cmd
2019-12-13 08:57:46,901.901 INFO:__main__:    proc = self.controller.run([os.path.join(BIN_PREFIX, "ceph")] + list(args), **kwargs)
2019-12-13 08:57:46,901.901 INFO:__main__:  File "../qa/tasks/vstart_runner.py", line 389, in run
2019-12-13 08:57:46,901.901 INFO:__main__:    proc.wait()
2019-12-13 08:57:46,901.901 INFO:__main__:  File "../qa/tasks/vstart_runner.py", line 189, in wait
2019-12-13 08:57:46,901.901 INFO:__main__:    raise CommandFailedError(self.args, self.exitstatus)
2019-12-13 08:57:46,902.902 INFO:__main__:CommandFailedError: Command failed with status 22: ['./bin/ceph', 'mgr', 'self-test', 'module', 'diskprediction_local']
2019-12-13 08:57:46,902.902 INFO:__main__:
2019-12-13 08:57:46,902.902 INFO:__main__:----------------------------------------------------------------------
2019-12-13 08:57:46,902.902 INFO:__main__:Ran 229 tests in 2590.925s
2019-12-13 08:57:46,903.903 INFO:__main__:
2019-12-13 08:57:46,903.903 INFO:__main__:FAILED (errors=1)
2019-12-13 08:57:46,903.903 INFO:__main__:
2019-12-13 08:57:46,903.903 INFO:__main__:======================================================================
2019-12-13 08:57:46,903.903 INFO:__main__:ERROR: test_diskprediction_local (tasks.mgr.test_module_selftest.TestModuleSelftest)
2019-12-13 08:57:46,904.904 INFO:__main__:----------------------------------------------------------------------
2019-12-13 08:57:46,904.904 INFO:__main__:Traceback (most recent call last):
2019-12-13 08:57:46,904.904 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/test_module_selftest.py", line 51, in test_diskprediction_local
2019-12-13 08:57:46,904.904 INFO:__main__:    self._selftest_plugin("diskprediction_local")
2019-12-13 08:57:46,904.904 INFO:__main__:  File "/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa/tasks/mgr/test_module_selftest.py", line 34, in _selftest_plugin
2019-12-13 08:57:46,905.905 INFO:__main__:    "mgr", "self-test", "module", module_name)
2019-12-13 08:57:46,905.905 INFO:__main__:  File "../qa/tasks/vstart_runner.py", line 936, in raw_cluster_cmd
2019-12-13 08:57:46,905.905 INFO:__main__:    proc = self.controller.run([os.path.join(BIN_PREFIX, "ceph")] + list(args), **kwargs)
2019-12-13 08:57:46,905.905 INFO:__main__:  File "../qa/tasks/vstart_runner.py", line 389, in run
2019-12-13 08:57:46,905.905 INFO:__main__:    proc.wait()
2019-12-13 08:57:46,905.905 INFO:__main__:  File "../qa/tasks/vstart_runner.py", line 189, in wait
2019-12-13 08:57:46,906.906 INFO:__main__:    raise CommandFailedError(self.args, self.exitstatus)
2019-12-13 08:57:46,906.906 INFO:__main__:CommandFailedError: Command failed with status 22: ['./bin/ceph', 'mgr', 'self-test', 'module', 'diskprediction_local']
2019-12-13 08:57:46,906.906 INFO:__main__:
Using guessed paths /home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/build/lib/ ['/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/qa', '/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/build/lib/cython_modules/lib.2', '/home/jenkins-build/build/workspace/ceph-dashboard-pr-backend/src/pybind']

#8 Updated by Neha Ojha about 1 year ago

  • Priority changed from Normal to High

#9 Updated by Laura Paduano 12 months ago

  • Status changed from New to Resolved
  • Assignee set to Sage Weil

Also available in: Atom PDF