Project

General

Profile

Bug #37290

mgr/dashboard: Failing QA test: test_safe_to_destroy

Added by Patrick Seidensal over 5 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
Normal
Category:
Testing & QA
Target version:
% Done:

100%

Source:
Tags:
Backport:
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Excerpt:


2018-11-16T13:28:44.423 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi204.front.sepia.ceph.com:7790/api/osd/13/safe_to_destroy
2018-11-16T13:28:44.435 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi204.front.sepia.ceph.com:7790/api/osd/0/safe_to_destroy
2018-11-16T13:28:44.436 INFO:tasks.ceph.mgr.y.smithi204.stdout:::ffff:172.21.0.51 - - [16/Nov/2018:13:28:44] "GET /api/osd/13/safe_to_destroy HTTP/1.1" 200 45 "" "python-requests/2.12.5" 
2018-11-16T13:28:44.444 INFO:tasks.ceph.mgr.y.smithi204.stdout:::ffff:172.21.0.51 - - [16/Nov/2018:13:28:44] "GET /api/osd/0/safe_to_destroy HTTP/1.1" 200 45 "" "python-requests/2.12.5" 
2018-11-16T13:28:44.445 DEBUG:tasks.ceph_test_case:wait_until_equal: True != False, waiting...
2018-11-16T13:28:46.961 INFO:teuthology.orchestra.run.smithi154:Running:
2018-11-16T13:28:46.961 INFO:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf
2018-11-16T13:28:46.965 INFO:teuthology.orchestra.run.smithi204:Running:
2018-11-16T13:28:46.965 INFO:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf
2018-11-16T13:28:49.445 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi204.front.sepia.ceph.com:7790/api/osd/0/safe_to_destroy
2018-11-16T13:28:49.456 INFO:tasks.ceph.mgr.y.smithi204.stdout:::ffff:172.21.0.51 - - [16/Nov/2018:13:28:49] "GET /api/osd/0/safe_to_destroy HTTP/1.1" 200 45 "" "python-requests/2.12.5" 
2018-11-16T13:28:49.457 DEBUG:tasks.ceph_test_case:wait_until_equal: True != False, waiting...
2018-11-16T13:28:54.458 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi204.front.sepia.ceph.com:7790/api/osd/0/safe_to_destroy
2018-11-16T13:28:54.470 DEBUG:tasks.ceph_test_case:wait_until_equal: True != False, waiting...
2018-11-16T13:28:54.470 INFO:tasks.ceph.mgr.y.smithi204.stdout:::ffff:172.21.0.51 - - [16/Nov/2018:13:28:54] "GET /api/osd/0/safe_to_destroy HTTP/1.1" 200 45 "" "python-requests/2.12.5" 
2018-11-16T13:28:59.471 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi204.front.sepia.ceph.com:7790/api/osd/0/safe_to_destroy
2018-11-16T13:28:59.483 DEBUG:tasks.ceph_test_case:wait_until_equal: True != False, waiting...
2018-11-16T13:28:59.483 INFO:tasks.ceph.mgr.y.smithi204.stdout:::ffff:172.21.0.51 - - [16/Nov/2018:13:28:59] "GET /api/osd/0/safe_to_destroy HTTP/1.1" 200 45 "" "python-requests/2.12.5" 
2018-11-16T13:29:04.485 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi204.front.sepia.ceph.com:7790/api/osd/0/safe_to_destroy
2018-11-16T13:29:04.501 INFO:tasks.ceph.mgr.y.smithi204.stdout:::ffff:172.21.0.51 - - [16/Nov/2018:13:29:04] "GET /api/osd/0/safe_to_destroy HTTP/1.1" 200 45 "" "python-requests/2.12.5" 
2018-11-16T13:29:04.503 DEBUG:tasks.ceph_test_case:wait_until_equal: True != False, waiting...
2018-11-16T13:29:09.505 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi204.front.sepia.ceph.com:7790/api/osd/0/safe_to_destroy
2018-11-16T13:29:09.519 DEBUG:tasks.ceph_test_case:wait_until_equal: True != False, waiting...
2018-11-16T13:29:09.520 INFO:tasks.ceph.mgr.y.smithi204.stdout:::ffff:172.21.0.51 - - [16/Nov/2018:13:29:09] "GET /api/osd/0/safe_to_destroy HTTP/1.1" 200 45 "" "python-requests/2.12.5" 
2018-11-16T13:29:14.520 INFO:tasks.mgr.dashboard.helper:Request GET to https://smithi204.front.sepia.ceph.com:7790/api/osd/0/safe_to_destroy
2018-11-16T13:29:14.531 INFO:tasks.ceph.mgr.y.smithi204.stdout:::ffff:172.21.0.51 - - [16/Nov/2018:13:29:14] "GET /api/osd/0/safe_to_destroy HTTP/1.1" 200 45 "" "python-requests/2.12.5" 
2018-11-16T13:29:14.532 INFO:tasks.cephfs_test_runner:test_safe_to_destroy (tasks.mgr.dashboard.test_osd.OsdTest) ... ERROR
2018-11-16T13:29:14.533 INFO:tasks.mgr.dashboard.helper:Request POST to https://smithi204.front.sepia.ceph.com:7790/api/osd/0/mark_in
2018-11-16T13:29:14.540 INFO:tasks.ceph.mgr.y.smithi204.stdout:::ffff:172.21.0.51 - - [16/Nov/2018:13:29:14] "POST /api/osd/0/mark_in HTTP/1.1" 200 24 "" "python-requests/2.12.5" 
2018-11-16T13:29:14.541 INFO:tasks.cephfs_test_runner:
2018-11-16T13:29:14.541 INFO:tasks.cephfs_test_runner:======================================================================
2018-11-16T13:29:14.541 INFO:tasks.cephfs_test_runner:ERROR: test_safe_to_destroy (tasks.mgr.dashboard.test_osd.OsdTest)
2018-11-16T13:29:14.541 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2018-11-16T13:29:14.541 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2018-11-16T13:29:14.541 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-tdehler-testing-25127/qa/tasks/mgr/dashboard/test_osd.py", line 112, in test_safe_to_destroy
2018-11-16T13:29:14.542 INFO:tasks.cephfs_test_runner:    self.wait_until_equal(get_destroy_status, False, 30)
2018-11-16T13:29:14.542 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-tdehler-testing-25127/qa/tasks/ceph_test_case.py", line 131, in wait_until_equal
2018-11-16T13:29:14.542 INFO:tasks.cephfs_test_runner:    elapsed, expect_val, val
2018-11-16T13:29:14.542 INFO:tasks.cephfs_test_runner:RuntimeError: Timed out after 30 seconds waiting for False (currently True)
2018-11-16T13:29:14.542 INFO:tasks.cephfs_test_runner:
2018-11-16T13:29:14.542 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2018-11-16T13:29:14.542 INFO:tasks.cephfs_test_runner:Ran 40 tests in 429.699s
2018-11-16T13:29:14.542 INFO:tasks.cephfs_test_runner:
2018-11-16T13:29:14.543 INFO:tasks.cephfs_test_runner:FAILED (errors=1)
2018-11-16T13:29:14.543 INFO:tasks.cephfs_test_runner:
2018-11-16T13:29:14.543 INFO:tasks.cephfs_test_runner:======================================================================
2018-11-16T13:29:14.543 INFO:tasks.cephfs_test_runner:ERROR: test_safe_to_destroy (tasks.mgr.dashboard.test_osd.OsdTest)
2018-11-16T13:29:14.543 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2018-11-16T13:29:14.543 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2018-11-16T13:29:14.543 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-tdehler-testing-25127/qa/tasks/mgr/dashboard/test_osd.py", line 112, in test_safe_to_destroy
2018-11-16T13:29:14.544 INFO:tasks.cephfs_test_runner:    self.wait_until_equal(get_destroy_status, False, 30)
2018-11-16T13:29:14.544 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-tdehler-testing-25127/qa/tasks/ceph_test_case.py", line 131, in wait_until_equal
2018-11-16T13:29:14.544 INFO:tasks.cephfs_test_runner:    elapsed, expect_val, val
2018-11-16T13:29:14.544 INFO:tasks.cephfs_test_runner:RuntimeError: Timed out after 30 seconds waiting for False (currently True)
2018-11-16T13:29:14.544 INFO:tasks.cephfs_test_runner:
2018-11-16T13:29:14.544 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 89, in run_tasks
    manager.__enter__()
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-tdehler-testing-25127/qa/tasks/cephfs_test_runner.py", line 207, in task
    raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests)))
RuntimeError: Test failure: test_safe_to_destroy (tasks.mgr.dashboard.test_osd.OsdTest)
2018-11-16T13:29:14.545 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner
2018-11-16T13:29:14.559 DEBUG:teuthology.run_tasks:Unwinding manager rgw
2018-11-16T13:29:14.572 ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 32, in nested
    yield vars
  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-tdehler-testing-25127/qa/tasks/rgw.py", line 313, in task
    yield
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 89, in run_tasks
    manager.__enter__()
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-tdehler-testing-25127/qa/tasks/cephfs_test_runner.py", line 207, in task
    raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests)))
RuntimeError: Test failure: test_safe_to_destroy (tasks.mgr.dashboard.test_osd.OsdTest)
2018-11-16T13:29:14.573 DEBUG:tasks.rgw.client.0:waiting for process to exit
2018-11-16T13:29:14.573 INFO:teuthology.orchestra.run:waiting for 300

More: http://pulpito.ceph.com/tdehler-2018-11-16_13:06:24-rados:mgr-wip-tdehler-testing-25127-distro-basic-smithi/

Might be related with: https://github.com/ceph/ceph/pull/24799#issuecomment-439286564

History

#1 Updated by Lenz Grimmer over 5 years ago

  • Subject changed from Failing QA test: test_safe_to_destroy to mgr/dashboard: Failing QA test: test_safe_to_destroy
  • Target version set to v14.0.0
  • Pull request ID set to 25127

#2 Updated by Patrick Seidensal over 5 years ago

  • Regression changed from Yes to No

#4 Updated by Lenz Grimmer over 5 years ago

  • Pull request ID changed from 25127 to 25149

#5 Updated by Lenz Grimmer over 5 years ago

  • Status changed from In Progress to Fix Under Review

#6 Updated by Patrick Seidensal over 5 years ago

  • % Done changed from 0 to 90

#7 Updated by Lenz Grimmer over 5 years ago

  • Status changed from Fix Under Review to Resolved

#8 Updated by Patrick Seidensal over 5 years ago

  • % Done changed from 90 to 100

#9 Updated by Ernesto Puerta almost 3 years ago

  • Project changed from mgr to Dashboard
  • Category changed from 151 to Testing & QA

Also available in: Atom PDF