Actions
Bug #37290
closedmgr/dashboard: Failing QA test: test_safe_to_destroy
Status:
Resolved
Priority:
Normal
Assignee:
Category:
Testing & QA
Target version:
% Done:
100%
Source:
Tags:
Backport:
Regression:
No
Severity:
2 - major
Reviewed:
Description
Excerpt:
2018-11-16T13:28:44.423 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi204.front.sepia.ceph.com:7790/api/osd/13/safe_to_destroy 2018-11-16T13:28:44.435 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi204.front.sepia.ceph.com:7790/api/osd/0/safe_to_destroy 2018-11-16T13:28:44.436 INFO:tasks.ceph.mgr.y.smithi204.stdout:::ffff:172.21.0.51 - - [16/Nov/2018:13:28:44] "GET /api/osd/13/safe_to_destroy HTTP/1.1" 200 45 "" "python-requests/2.12.5" 2018-11-16T13:28:44.444 INFO:tasks.ceph.mgr.y.smithi204.stdout:::ffff:172.21.0.51 - - [16/Nov/2018:13:28:44] "GET /api/osd/0/safe_to_destroy HTTP/1.1" 200 45 "" "python-requests/2.12.5" 2018-11-16T13:28:44.445 DEBUG:tasks.ceph_test_case:wait_until_equal: True != False, waiting... 2018-11-16T13:28:46.961 INFO:teuthology.orchestra.run.smithi154:Running: 2018-11-16T13:28:46.961 INFO:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2018-11-16T13:28:46.965 INFO:teuthology.orchestra.run.smithi204:Running: 2018-11-16T13:28:46.965 INFO:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2018-11-16T13:28:49.445 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi204.front.sepia.ceph.com:7790/api/osd/0/safe_to_destroy 2018-11-16T13:28:49.456 INFO:tasks.ceph.mgr.y.smithi204.stdout:::ffff:172.21.0.51 - - [16/Nov/2018:13:28:49] "GET /api/osd/0/safe_to_destroy HTTP/1.1" 200 45 "" "python-requests/2.12.5" 2018-11-16T13:28:49.457 DEBUG:tasks.ceph_test_case:wait_until_equal: True != False, waiting... 2018-11-16T13:28:54.458 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi204.front.sepia.ceph.com:7790/api/osd/0/safe_to_destroy 2018-11-16T13:28:54.470 DEBUG:tasks.ceph_test_case:wait_until_equal: True != False, waiting... 2018-11-16T13:28:54.470 INFO:tasks.ceph.mgr.y.smithi204.stdout:::ffff:172.21.0.51 - - [16/Nov/2018:13:28:54] "GET /api/osd/0/safe_to_destroy HTTP/1.1" 200 45 "" "python-requests/2.12.5" 2018-11-16T13:28:59.471 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi204.front.sepia.ceph.com:7790/api/osd/0/safe_to_destroy 2018-11-16T13:28:59.483 DEBUG:tasks.ceph_test_case:wait_until_equal: True != False, waiting... 2018-11-16T13:28:59.483 INFO:tasks.ceph.mgr.y.smithi204.stdout:::ffff:172.21.0.51 - - [16/Nov/2018:13:28:59] "GET /api/osd/0/safe_to_destroy HTTP/1.1" 200 45 "" "python-requests/2.12.5" 2018-11-16T13:29:04.485 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi204.front.sepia.ceph.com:7790/api/osd/0/safe_to_destroy 2018-11-16T13:29:04.501 INFO:tasks.ceph.mgr.y.smithi204.stdout:::ffff:172.21.0.51 - - [16/Nov/2018:13:29:04] "GET /api/osd/0/safe_to_destroy HTTP/1.1" 200 45 "" "python-requests/2.12.5" 2018-11-16T13:29:04.503 DEBUG:tasks.ceph_test_case:wait_until_equal: True != False, waiting... 2018-11-16T13:29:09.505 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi204.front.sepia.ceph.com:7790/api/osd/0/safe_to_destroy 2018-11-16T13:29:09.519 DEBUG:tasks.ceph_test_case:wait_until_equal: True != False, waiting... 2018-11-16T13:29:09.520 INFO:tasks.ceph.mgr.y.smithi204.stdout:::ffff:172.21.0.51 - - [16/Nov/2018:13:29:09] "GET /api/osd/0/safe_to_destroy HTTP/1.1" 200 45 "" "python-requests/2.12.5" 2018-11-16T13:29:14.520 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi204.front.sepia.ceph.com:7790/api/osd/0/safe_to_destroy 2018-11-16T13:29:14.531 INFO:tasks.ceph.mgr.y.smithi204.stdout:::ffff:172.21.0.51 - - [16/Nov/2018:13:29:14] "GET /api/osd/0/safe_to_destroy HTTP/1.1" 200 45 "" "python-requests/2.12.5" 2018-11-16T13:29:14.532 INFO:tasks.cephfs_test_runner:test_safe_to_destroy (tasks.mgr.dashboard.test_osd.OsdTest) ... ERROR 2018-11-16T13:29:14.533 INFO:tasks.mgr.dashboard.helper:Request POST to https://smithi204.front.sepia.ceph.com:7790/api/osd/0/mark_in 2018-11-16T13:29:14.540 INFO:tasks.ceph.mgr.y.smithi204.stdout:::ffff:172.21.0.51 - - [16/Nov/2018:13:29:14] "POST /api/osd/0/mark_in HTTP/1.1" 200 24 "" "python-requests/2.12.5" 2018-11-16T13:29:14.541 INFO:tasks.cephfs_test_runner: 2018-11-16T13:29:14.541 INFO:tasks.cephfs_test_runner:====================================================================== 2018-11-16T13:29:14.541 INFO:tasks.cephfs_test_runner:ERROR: test_safe_to_destroy (tasks.mgr.dashboard.test_osd.OsdTest) 2018-11-16T13:29:14.541 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2018-11-16T13:29:14.541 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2018-11-16T13:29:14.541 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-tdehler-testing-25127/qa/tasks/mgr/dashboard/test_osd.py", line 112, in test_safe_to_destroy 2018-11-16T13:29:14.542 INFO:tasks.cephfs_test_runner: self.wait_until_equal(get_destroy_status, False, 30) 2018-11-16T13:29:14.542 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-tdehler-testing-25127/qa/tasks/ceph_test_case.py", line 131, in wait_until_equal 2018-11-16T13:29:14.542 INFO:tasks.cephfs_test_runner: elapsed, expect_val, val 2018-11-16T13:29:14.542 INFO:tasks.cephfs_test_runner:RuntimeError: Timed out after 30 seconds waiting for False (currently True) 2018-11-16T13:29:14.542 INFO:tasks.cephfs_test_runner: 2018-11-16T13:29:14.542 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2018-11-16T13:29:14.542 INFO:tasks.cephfs_test_runner:Ran 40 tests in 429.699s 2018-11-16T13:29:14.542 INFO:tasks.cephfs_test_runner: 2018-11-16T13:29:14.543 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2018-11-16T13:29:14.543 INFO:tasks.cephfs_test_runner: 2018-11-16T13:29:14.543 INFO:tasks.cephfs_test_runner:====================================================================== 2018-11-16T13:29:14.543 INFO:tasks.cephfs_test_runner:ERROR: test_safe_to_destroy (tasks.mgr.dashboard.test_osd.OsdTest) 2018-11-16T13:29:14.543 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2018-11-16T13:29:14.543 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2018-11-16T13:29:14.543 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-tdehler-testing-25127/qa/tasks/mgr/dashboard/test_osd.py", line 112, in test_safe_to_destroy 2018-11-16T13:29:14.544 INFO:tasks.cephfs_test_runner: self.wait_until_equal(get_destroy_status, False, 30) 2018-11-16T13:29:14.544 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-tdehler-testing-25127/qa/tasks/ceph_test_case.py", line 131, in wait_until_equal 2018-11-16T13:29:14.544 INFO:tasks.cephfs_test_runner: elapsed, expect_val, val 2018-11-16T13:29:14.544 INFO:tasks.cephfs_test_runner:RuntimeError: Timed out after 30 seconds waiting for False (currently True) 2018-11-16T13:29:14.544 INFO:tasks.cephfs_test_runner: 2018-11-16T13:29:14.544 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 89, in run_tasks manager.__enter__() File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-tdehler-testing-25127/qa/tasks/cephfs_test_runner.py", line 207, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_safe_to_destroy (tasks.mgr.dashboard.test_osd.OsdTest) 2018-11-16T13:29:14.545 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2018-11-16T13:29:14.559 DEBUG:teuthology.run_tasks:Unwinding manager rgw 2018-11-16T13:29:14.572 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-tdehler-testing-25127/qa/tasks/rgw.py", line 313, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 89, in run_tasks manager.__enter__() File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-tdehler-testing-25127/qa/tasks/cephfs_test_runner.py", line 207, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_safe_to_destroy (tasks.mgr.dashboard.test_osd.OsdTest) 2018-11-16T13:29:14.573 DEBUG:tasks.rgw.client.0:waiting for process to exit 2018-11-16T13:29:14.573 INFO:teuthology.orchestra.run:waiting for 300
Might be related with: https://github.com/ceph/ceph/pull/24799#issuecomment-439286564
Actions