Project

General

Profile

Actions
Copy link

Bug #40618

closed

Test failure: test_osd_came_back (tasks.mgr.test_progress.TestProgress)

Added by Kefu Chai almost 5 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Ricardo Dias
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
30095
Crash signature (v1):
Crash signature (v2):

Description

2019-07-02T06:37:28.073 INFO:tasks.cephfs_test_runner:FAIL: test_osd_came_back (tasks.mgr.test_progress.TestProgress)
2019-07-02T06:37:28.073 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2019-07-02T06:37:28.073 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2019-07-02T06:37:28.073 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2019-06-28-1816/qa/tasks/mgr/test_progress.py", line 176, in test_osd_came_back
2019-07-02T06:37:28.073 INFO:tasks.cephfs_test_runner:    self.assertTrue(self._is_quiet())
2019-07-02T06:37:28.073 INFO:tasks.cephfs_test_runner:AssertionError: False is not true
2019-07-02T06:37:28.073 INFO:tasks.cephfs_test_runner:
2019-07-02T06:37:28.074 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2019-07-02T06:37:28.074 INFO:tasks.cephfs_test_runner:Ran 1 test in 71.945s
2019-07-02T06:37:28.074 INFO:tasks.cephfs_test_runner:
2019-07-02T06:37:28.074 INFO:tasks.cephfs_test_runner:FAILED (failures=1)

2019-07-02T06:37:25.213 INFO:teuthology.orchestra.run.mira030.stderr:2019-07-02T06:37:25.206+0000 7fa7c6397700  1 -- 172.21.4.116:0/1613632094 --> [v2:172.21.4.116:6800/14006,v1:172.21.4.116:6801/14006] -- command(tid 0: {"prefix": "progress json", "target": ["mgr", ""]})
 v1 -- 0x7fa7c016e030 con 0x7fa7a4033b00
2019-07-02T06:37:25.214 INFO:teuthology.orchestra.run.mira030.stderr:2019-07-02T06:37:25.210+0000 7fa7bcff9700  1 -- 172.21.4.116:0/1613632094 <== mgr.4620 v2:172.21.4.116:6800/14006 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+434 (crc 0 0 0) 0x7fa7b0000fb0 con 0x7fa7a403
3b00
2019-07-02T06:37:25.214 INFO:teuthology.orchestra.run.mira030.stderr:2019-07-02T06:37:25.210+0000 7fa7bcff9700 20 mgrc handle_command_reply command_reply(tid 0: 0 ) v1
2019-07-02T06:37:25.216 INFO:teuthology.orchestra.run.mira030.stdout:{
2019-07-02T06:37:25.216 INFO:teuthology.orchestra.run.mira030.stdout:  "completed": [
2019-07-02T06:37:25.216 INFO:teuthology.orchestra.run.mira030.stdout:    {
2019-07-02T06:37:25.216 INFO:teuthology.orchestra.run.mira030.stdout:      "message": "Rebalancing after osd.0 marked out",
2019-07-02T06:37:25.216 INFO:teuthology.orchestra.run.mira030.stdout:      "id": "6aec1958-769a-44b5-bd49-ecd619eb65af",
2019-07-02T06:37:25.216 INFO:teuthology.orchestra.run.mira030.stdout:      "refs": [
2019-07-02T06:37:25.216 INFO:teuthology.orchestra.run.mira030.stdout:        [
2019-07-02T06:37:25.217 INFO:teuthology.orchestra.run.mira030.stdout:          "osd",
2019-07-02T06:37:25.217 INFO:teuthology.orchestra.run.mira030.stdout:          0
2019-07-02T06:37:25.217 INFO:teuthology.orchestra.run.mira030.stdout:        ]
2019-07-02T06:37:25.217 INFO:teuthology.orchestra.run.mira030.stdout:      ]
2019-07-02T06:37:25.217 INFO:teuthology.orchestra.run.mira030.stdout:    }
2019-07-02T06:37:25.217 INFO:teuthology.orchestra.run.mira030.stdout:  ],
2019-07-02T06:37:25.217 INFO:teuthology.orchestra.run.mira030.stdout:  "events": [
2019-07-02T06:37:25.218 INFO:teuthology.orchestra.run.mira030.stdout:    {
2019-07-02T06:37:25.218 INFO:teuthology.orchestra.run.mira030.stdout:      "message": "Rebalancing after osd.0 marked in",
2019-07-02T06:37:25.218 INFO:teuthology.orchestra.run.mira030.stdout:      "id": "2416d7fe-52e8-4ab8-98ac-0b307388e2fd",
2019-07-02T06:37:25.218 INFO:teuthology.orchestra.run.mira030.stdout:      "refs": [
2019-07-02T06:37:25.218 INFO:teuthology.orchestra.run.mira030.stdout:        [
2019-07-02T06:37:25.218 INFO:teuthology.orchestra.run.mira030.stdout:          "osd",
2019-07-02T06:37:25.218 INFO:teuthology.orchestra.run.mira030.stdout:          0
2019-07-02T06:37:25.219 INFO:teuthology.orchestra.run.mira030.stdout:        ]
2019-07-02T06:37:25.219 INFO:teuthology.orchestra.run.mira030.stdout:      ]
2019-07-02T06:37:25.219 INFO:teuthology.orchestra.run.mira030.stdout:    }
2019-07-02T06:37:25.219 INFO:teuthology.orchestra.run.mira030.stdout:  ]
2019-07-02T06:37:25.219 INFO:teuthology.orchestra.run.mira030.stdout:}

see

Related issues 2 (0 open — 2 closed)

Has duplicate RADOS - Bug #40825: test_osd_came_back (tasks.mgr.test_progress.TestProgress) ... FAILDuplicate07/18/2019

Actions
Has duplicate mgr - Bug #41400: Test failure: test_osd_came_back (tasks.mgr.test_progress.TestProgress)Duplicate08/22/2019

Actions
Actions
Copy link
 #2

Updated by Kefu Chai almost 5 years ago

  • Status changed from 12 to Fix Under Review
  • Pull request ID set to 28840
Actions
Copy link
 #3

Updated by Sage Weil almost 5 years ago

  • Status changed from Fix Under Review to Resolved
Actions
Copy link
 #4

Updated by Kefu Chai over 4 years ago

  • Has duplicate Bug #40825: test_osd_came_back (tasks.mgr.test_progress.TestProgress) ... FAIL added
Actions
Copy link
 #5

Updated by Kefu Chai over 4 years ago

  • Status changed from Resolved to In Progress
Actions
Copy link
 #7

Updated by Kefu Chai over 4 years ago

http://pulpito.ceph.com/kchai-2019-08-12_14:45:48-rados-wip-kefu-testing-2019-08-12-1306-distro-basic-mira/4211314/

still broken

the tested branch includes the PRs above.

2019-08-12T17:37:10.800 INFO:tasks.cephfs_test_runner:======================================================================
2019-08-12T17:37:10.800 INFO:tasks.cephfs_test_runner:ERROR: test_osd_came_back (tasks.mgr.test_progress.TestProgress)
2019-08-12T17:37:10.800 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2019-08-12T17:37:10.800 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2019-08-12T17:37:10.800 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2019-08-12-1306/qa/tasks/mgr/test_progress.py", line 202, in test_osd_came_back
2019-08-12T17:37:10.801 INFO:tasks.cephfs_test_runner:    ev2 = self._simulate_back_in([0], ev1)
2019-08-12T17:37:10.801 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2019-08-12-1306/qa/tasks/mgr/test_progress.py", line 128, in _simulate_back_in
2019-08-12T17:37:10.801 INFO:tasks.cephfs_test_runner:    timeout=self.EVENT_CREATION_PERIOD)
2019-08-12T17:37:10.801 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2019-08-12-1306/qa/tasks/ceph_test_case.py", line 145, in wait_until_equal
2019-08-12T17:37:10.801 INFO:tasks.cephfs_test_runner:    elapsed, expect_val, val
2019-08-12T17:37:10.801 INFO:tasks.cephfs_test_runner:RuntimeError: Timed out after 5 seconds waiting for 1 (currently 0)
2019-08-12T17:37:10.802 INFO:tasks.cephfs_test_runner:
2019-08-12T17:37:10.802 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2019-08-12T17:37:10.802 INFO:tasks.cephfs_test_runner:Ran 1 test in 92.563s
2019-08-12T17:37:10.802 INFO:tasks.cephfs_test_runner:
2019-08-12T17:37:10.802 INFO:tasks.cephfs_test_runner:FAILED (errors=1)
Actions
Copy link
 #8

Updated by Kefu Chai over 4 years ago

please note, https://github.com/ceph/ceph/pull/29625 does not address this issue.

Actions
Copy link
 #9

Updated by Neha Ojha over 4 years ago

  • Is duplicate of Bug #41400: Test failure: test_osd_came_back (tasks.mgr.test_progress.TestProgress) added
Actions
Copy link
 #10

Updated by Neha Ojha over 4 years ago

  • Priority changed from Normal to Urgent
Actions
Copy link
 #11

Updated by Kefu Chai over 4 years ago

  • Is duplicate of deleted (Bug #41400: Test failure: test_osd_came_back (tasks.mgr.test_progress.TestProgress))
Actions
Copy link
 #12

Updated by Kefu Chai over 4 years ago

  • Has duplicate Bug #41400: Test failure: test_osd_came_back (tasks.mgr.test_progress.TestProgress) added
Actions
Copy link
 #13

Updated by Ricardo Dias over 4 years ago

  • Assignee set to Ricardo Dias
  • Pull request ID changed from 28840 to 30095

Opened PR to fix QA test failures. https://github.com/ceph/ceph/pull/30095

Actions
Copy link
 #14

Updated by Kefu Chai over 4 years ago

  • Status changed from In Progress to Resolved
Actions
Copy link

Also available in: Atom PDF