Bug #38268
closedmgr/dashboard/qa: OSD tests get stuck on Teuthology
0%
Description
The last few days I scheduled several tests on Teuthology:
- http://pulpito.ceph.com/tdehler-2019-02-08_17:01:41-rados:mgr-wip-tdehler-testing-25233-distro-basic-smithi/
- http://pulpito.ceph.com/tdehler-2019-02-11_10:46:11-rados:mgr-wip-tdehler-testing-25233-distro-basic-smithi/
- http://pulpito.ceph.com/tdehler-2019-02-08_11:43:27-rados:mgr-wip-tdehler-testing-25989-distro-basic-smithi/
- http://pulpito.ceph.com/tdehler-2019-02-11_10:56:55-rados:mgr-wip-tdehler-testing-25989-distro-basic-smithi/
Besides https://tracker.ceph.com/issues/38255 some of the jobs get stuck in OSD test cases which end up in status "dead". I had a look at the logfiles and saw each job hanging in e.g.:
2019-02-11T22:57:27.618 INFO:tasks.ceph.mon.b.smithi142.stderr:2019-02-11 22:57:27.616 7f7f0a837700 -1 mon.b@1(peon) e1 get_health_metrics reporting 1 slow ops, oldest is mon_command({"prefix": "osd safe-to-destroy", "target": ["mgr", ""], "ids": ["13"], "format": "json"} v 0)
Which could be related to test_osd.test_safe_to_destroy test case, so I removed the test_safe_to_destroy test case built packages and started the tests again. The test job is (besides https://tracker.ceph.com/issues/3825 and https://tracker.ceph.com/issues/38265) hanging in
2019-02-12T08:53:29.649 INFO:tasks.ceph.mon.a.smithi198.stderr:2019-02-12 08:53:29.638 7fa3ffcfa700 -1 mon.a@1(peon) e1 get_health_metrics reporting 1 slow ops, oldest is mon_command({"prefix": "osd scrub", "who": "0", "format": "json"} v 0)
Updated by Lenz Grimmer about 5 years ago
- Status changed from New to Fix Under Review
- Assignee set to Ricardo Dias
- Pull request ID set to 26385
Updated by Lenz Grimmer about 5 years ago
- Status changed from Fix Under Review to Resolved
PR#26385 has been merged, hopefully this issue has been resolved. Please re-open, if you still observe these symptoms.
Updated by Ernesto Puerta about 3 years ago
- Project changed from mgr to Dashboard
- Category changed from 151 to Testing & QA