Project

General

Profile

Bug #42512

mgr/dashboard: tasks.mgr.dashboard.test_rbd_mirroring suite is failing

Added by Tatjana Dehler 11 months ago. Updated 10 months ago.

Status:
Resolved
Priority:
Normal
Category:
dashboard/qa
Target version:
% Done:

0%

Source:
Community (dev)
Tags:
dashboard, qa
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

When running (by https://github.com/ceph/ceph/blob/master/src/pybind/mgr/dashboard/run-backend-api-tests.sh) tasks.mgr.dashboard.test_rbd_mirroring the following tests are failing on my docker environment:

2019-10-25 13:57:01,345.345 INFO:__main__:======================================================================
2019-10-25 13:57:01,345.345 INFO:__main__:ERROR: test_create_access_permissions (tasks.mgr.dashboard.test_rbd_mirroring.RbdMirroringTest)
2019-10-25 13:57:01,346.346 INFO:__main__:----------------------------------------------------------------------
2019-10-25 13:57:01,346.346 INFO:__main__:Traceback (most recent call last):
2019-10-25 13:57:01,346.346 INFO:__main__:  File "/ceph/qa/tasks/mgr/dashboard/helper.py", line 154, in setUp
2019-10-25 13:57:01,346.346 INFO:__main__:    self.wait_for_health_clear(20)
2019-10-25 13:57:01,346.346 INFO:__main__:  File "/ceph/qa/tasks/ceph_test_case.py", line 131, in wait_for_health_clear
2019-10-25 13:57:01,346.346 INFO:__main__:    self.wait_until_true(is_clear, timeout)
2019-10-25 13:57:01,346.346 INFO:__main__:  File "/ceph/qa/tasks/ceph_test_case.py", line 163, in wait_until_true
2019-10-25 13:57:01,347.347 INFO:__main__:    raise RuntimeError("Timed out after {0}s".format(elapsed))
2019-10-25 13:57:01,347.347 INFO:__main__:RuntimeError: Timed out after 20s
2019-10-25 13:57:01,347.347 INFO:__main__:
2019-10-25 13:57:01,347.347 INFO:__main__:----------------------------------------------------------------------

2019-10-25 14:13:12,242.242 INFO:__main__:======================================================================
2019-10-25 14:13:12,242.242 INFO:__main__:ERROR: test_delete_access_permissions (tasks.mgr.dashboard.test_rbd_mirroring.RbdMirroringTest)
2019-10-25 14:13:12,243.243 INFO:__main__:----------------------------------------------------------------------
2019-10-25 14:13:12,243.243 INFO:__main__:Traceback (most recent call last):
2019-10-25 14:13:12,243.243 INFO:__main__:  File "/ceph/qa/tasks/mgr/dashboard/helper.py", line 154, in setUp
2019-10-25 14:13:12,243.243 INFO:__main__:    self.wait_for_health_clear(20)
2019-10-25 14:13:12,243.243 INFO:__main__:  File "/ceph/qa/tasks/ceph_test_case.py", line 131, in wait_for_health_clear
2019-10-25 14:13:12,243.243 INFO:__main__:    self.wait_until_true(is_clear, timeout)
2019-10-25 14:13:12,243.243 INFO:__main__:  File "/ceph/qa/tasks/ceph_test_case.py", line 163, in wait_until_true
2019-10-25 14:13:12,243.243 INFO:__main__:    raise RuntimeError("Timed out after {0}s".format(elapsed))
2019-10-25 14:13:12,244.244 INFO:__main__:RuntimeError: Timed out after 20s
2019-10-25 14:13:12,244.244 INFO:__main__:
2019-10-25 14:13:12,244.244 INFO:__main__:----------------------------------------------------------------------

2019-10-25 14:19:50,192.192 INFO:__main__:======================================================================
2019-10-25 14:19:50,193.193 INFO:__main__:ERROR: test_disable_mirror_with_peers (tasks.mgr.dashboard.test_rbd_mirroring.RbdMirroringTest)
2019-10-25 14:19:50,193.193 INFO:__main__:----------------------------------------------------------------------
2019-10-25 14:19:50,193.193 INFO:__main__:Traceback (most recent call last):
2019-10-25 14:19:50,193.193 INFO:__main__:  File "/ceph/qa/tasks/mgr/dashboard/helper.py", line 154, in setUp
2019-10-25 14:19:50,193.193 INFO:__main__:    self.wait_for_health_clear(20)
2019-10-25 14:19:50,193.193 INFO:__main__:  File "/ceph/qa/tasks/ceph_test_case.py", line 131, in wait_for_health_clear
2019-10-25 14:19:50,193.193 INFO:__main__:    self.wait_until_true(is_clear, timeout)
2019-10-25 14:19:50,193.193 INFO:__main__:  File "/ceph/qa/tasks/ceph_test_case.py", line 163, in wait_until_true
2019-10-25 14:19:50,193.193 INFO:__main__:    raise RuntimeError("Timed out after {0}s".format(elapsed))
2019-10-25 14:19:50,194.194 INFO:__main__:RuntimeError: Timed out after 20s
2019-10-25 14:19:50,194.194 INFO:__main__:
2019-10-25 14:19:50,194.194 INFO:__main__:----------------------------------------------------------------------

2019-10-25 14:27:04,669.669 INFO:__main__:======================================================================
2019-10-25 14:27:04,669.669 INFO:__main__:ERROR: test_mirror_mode (tasks.mgr.dashboard.test_rbd_mirroring.RbdMirroringTest)
2019-10-25 14:27:04,669.669 INFO:__main__:----------------------------------------------------------------------
2019-10-25 14:27:04,669.669 INFO:__main__:Traceback (most recent call last):
2019-10-25 14:27:04,669.669 INFO:__main__:  File "/ceph/qa/tasks/mgr/dashboard/helper.py", line 154, in setUp
2019-10-25 14:27:04,670.670 INFO:__main__:    self.wait_for_health_clear(20)
2019-10-25 14:27:04,670.670 INFO:__main__:  File "/ceph/qa/tasks/ceph_test_case.py", line 131, in wait_for_health_clear
2019-10-25 14:27:04,670.670 INFO:__main__:    self.wait_until_true(is_clear, timeout)
2019-10-25 14:27:04,670.670 INFO:__main__:  File "/ceph/qa/tasks/ceph_test_case.py", line 163, in wait_until_true
2019-10-25 14:27:04,670.670 INFO:__main__:    raise RuntimeError("Timed out after {0}s".format(elapsed))
2019-10-25 14:27:04,670.670 INFO:__main__:RuntimeError: Timed out after 20s
2019-10-25 14:27:04,671.671 INFO:__main__:
2019-10-25 14:27:04,671.671 INFO:__main__:----------------------------------------------------------------------

2019-10-25 14:32:26,257.257 INFO:__main__:======================================================================
2019-10-25 14:32:26,258.258 INFO:__main__:ERROR: test_peer (tasks.mgr.dashboard.test_rbd_mirroring.RbdMirroringTest)
2019-10-25 14:32:26,258.258 INFO:__main__:----------------------------------------------------------------------
2019-10-25 14:32:26,258.258 INFO:__main__:Traceback (most recent call last):
2019-10-25 14:32:26,258.258 INFO:__main__:  File "/ceph/qa/tasks/mgr/dashboard/helper.py", line 154, in setUp
2019-10-25 14:32:26,259.259 INFO:__main__:    self.wait_for_health_clear(20)
2019-10-25 14:32:26,268.268 INFO:__main__:  File "/ceph/qa/tasks/ceph_test_case.py", line 131, in wait_for_health_clear
2019-10-25 14:32:26,269.269 INFO:__main__:    self.wait_until_true(is_clear, timeout)
2019-10-25 14:32:26,269.269 INFO:__main__:  File "/ceph/qa/tasks/ceph_test_case.py", line 163, in wait_until_true
2019-10-25 14:32:26,269.269 INFO:__main__:    raise RuntimeError("Timed out after {0}s".format(elapsed))
2019-10-25 14:32:26,269.269 INFO:__main__:RuntimeError: Timed out after 20s
2019-10-25 14:32:26,269.269 INFO:__main__:
2019-10-25 14:32:26,270.270 INFO:__main__:----------------------------------------------------------------------

2019-10-28 09:09:40,558.558 INFO:__main__:======================================================================
2019-10-28 09:09:40,558.558 INFO:__main__:ERROR: test_read_access_permissions (tasks.mgr.dashboard.test_rbd_mirroring.RbdMirroringTest)
2019-10-28 09:09:40,558.558 INFO:__main__:----------------------------------------------------------------------
2019-10-28 09:09:40,559.559 INFO:__main__:Traceback (most recent call last):
2019-10-28 09:09:40,559.559 INFO:__main__:  File "/ceph/qa/tasks/mgr/dashboard/helper.py", line 154, in setUp
2019-10-28 09:09:40,559.559 INFO:__main__:    self.wait_for_health_clear(20)
2019-10-28 09:09:40,559.559 INFO:__main__:  File "/ceph/qa/tasks/ceph_test_case.py", line 131, in wait_for_health_clear
2019-10-28 09:09:40,559.559 INFO:__main__:    self.wait_until_true(is_clear, timeout)
2019-10-28 09:09:40,559.559 INFO:__main__:  File "/ceph/qa/tasks/ceph_test_case.py", line 163, in wait_until_true
2019-10-28 09:09:40,559.559 INFO:__main__:    raise RuntimeError("Timed out after {0}s".format(elapsed))
2019-10-28 09:09:40,559.559 INFO:__main__:RuntimeError: Timed out after 20s
2019-10-28 09:09:40,560.560 INFO:__main__:
2019-10-28 09:09:40,560.560 INFO:__main__:----------------------------------------------------------------------

2019-10-28 09:13:51,844.844 INFO:__main__:======================================================================
2019-10-28 09:13:51,845.845 INFO:__main__:ERROR: test_set_invalid_mirror_mode (tasks.mgr.dashboard.test_rbd_mirroring.RbdMirroringTest)
2019-10-28 09:13:51,845.845 INFO:__main__:----------------------------------------------------------------------
2019-10-28 09:13:51,846.846 INFO:__main__:Traceback (most recent call last):
2019-10-28 09:13:51,847.847 INFO:__main__:  File "/ceph/qa/tasks/mgr/dashboard/helper.py", line 154, in setUp
2019-10-28 09:13:51,848.848 INFO:__main__:    self.wait_for_health_clear(20)
2019-10-28 09:13:51,848.848 INFO:__main__:  File "/ceph/qa/tasks/ceph_test_case.py", line 131, in wait_for_health_clear
2019-10-28 09:13:51,849.849 INFO:__main__:    self.wait_until_true(is_clear, timeout)
2019-10-28 09:13:51,851.851 INFO:__main__:  File "/ceph/qa/tasks/ceph_test_case.py", line 163, in wait_until_true
2019-10-28 09:13:51,851.851 INFO:__main__:    raise RuntimeError("Timed out after {0}s".format(elapsed))
2019-10-28 09:13:51,852.852 INFO:__main__:RuntimeError: Timed out after 20s
2019-10-28 09:13:51,852.852 INFO:__main__:
2019-10-28 09:13:51,853.853 INFO:__main__:----------------------------------------------------------------------

2019-10-28 09:19:57,907.907 INFO:__main__:======================================================================
2019-10-28 09:19:57,910.910 INFO:__main__:ERROR: test_set_same_mirror_mode (tasks.mgr.dashboard.test_rbd_mirroring.RbdMirroringTest)
2019-10-28 09:19:57,910.910 INFO:__main__:----------------------------------------------------------------------
2019-10-28 09:19:57,910.910 INFO:__main__:Traceback (most recent call last):
2019-10-28 09:19:57,911.911 INFO:__main__:  File "/ceph/qa/tasks/mgr/dashboard/helper.py", line 154, in setUp
2019-10-28 09:19:57,911.911 INFO:__main__:    self.wait_for_health_clear(20)
2019-10-28 09:19:57,911.911 INFO:__main__:  File "/ceph/qa/tasks/ceph_test_case.py", line 131, in wait_for_health_clear
2019-10-28 09:19:57,911.911 INFO:__main__:    self.wait_until_true(is_clear, timeout)
2019-10-28 09:19:57,912.912 INFO:__main__:  File "/ceph/qa/tasks/ceph_test_case.py", line 163, in wait_until_true
2019-10-28 09:19:57,912.912 INFO:__main__:    raise RuntimeError("Timed out after {0}s".format(elapsed))
2019-10-28 09:19:57,912.912 INFO:__main__:RuntimeError: Timed out after 20s
2019-10-28 09:19:57,912.912 INFO:__main__:
2019-10-28 09:19:57,914.914 INFO:__main__:----------------------------------------------------------------------

2019-10-28 09:25:02,801.801 INFO:__main__:======================================================================
2019-10-28 09:25:02,801.801 INFO:__main__:ERROR: test_update_access_permissions (tasks.mgr.dashboard.test_rbd_mirroring.RbdMirroringTest)
2019-10-28 09:25:02,801.801 INFO:__main__:----------------------------------------------------------------------
2019-10-28 09:25:02,801.801 INFO:__main__:Traceback (most recent call last):
2019-10-28 09:25:02,801.801 INFO:__main__:  File "/ceph/qa/tasks/mgr/dashboard/helper.py", line 154, in setUp
2019-10-28 09:25:02,801.801 INFO:__main__:    self.wait_for_health_clear(20)
2019-10-28 09:25:02,802.802 INFO:__main__:  File "/ceph/qa/tasks/ceph_test_case.py", line 131, in wait_for_health_clear
2019-10-28 09:25:02,802.802 INFO:__main__:    self.wait_until_true(is_clear, timeout)
2019-10-28 09:25:02,802.802 INFO:__main__:  File "/ceph/qa/tasks/ceph_test_case.py", line 163, in wait_until_true
2019-10-28 09:25:02,802.802 INFO:__main__:    raise RuntimeError("Timed out after {0}s".format(elapsed))
2019-10-28 09:25:02,802.802 INFO:__main__:RuntimeError: Timed out after 20s
2019-10-28 09:25:02,802.802 INFO:__main__:
2019-10-28 09:25:02,802.802 INFO:__main__:----------------------------------------------------------------------

Related issues

Related to mgr - Bug #42244: mgr/dashboard: Two QA suites are missing from dashboard.yaml Resolved
Related to mgr - Bug #42446: mgr/dashboard: tasks.mgr.dashboard.test_ganesha.GaneshaTest.test_create_export fails in ceph-dashboard-pr-backend New

History

#1 Updated by Tatjana Dehler 11 months ago

  • Related to Bug #42244: mgr/dashboard: Two QA suites are missing from dashboard.yaml added

#2 Updated by Tatjana Dehler 11 months ago

  • Description updated (diff)

#3 Updated by Tatjana Dehler 11 months ago

  • Description updated (diff)

#4 Updated by Tatjana Dehler 11 months ago

  • Related to Bug #42446: mgr/dashboard: tasks.mgr.dashboard.test_ganesha.GaneshaTest.test_create_export fails in ceph-dashboard-pr-backend added

#5 Updated by Tatjana Dehler 11 months ago

The issue seems indeed to be related to https://tracker.ceph.com/issues/42446:

2019-10-29 15:11:16,814.814 ERROR:tasks.ceph_test_case:{u'POOL_PG_NUM_NOT_POWER_OF_TWO': {u'muted': False, u'severity': u'HEALTH_WARN', u'summary': {u'count': 1, u'message': u'1 pool(s) have non-power-of-two pg_num'}}}

#6 Updated by Jason Dillaman 10 months ago

  • Status changed from New to Fix Under Review
  • Pull request ID set to 31062

#7 Updated by Laura Paduano 10 months ago

  • Status changed from Fix Under Review to Resolved
  • Assignee set to Jason Dillaman

#8 Updated by Laura Paduano 10 months ago

  • Target version set to v15.0.0

Also available in: Atom PDF