Project

General

Profile

Bug #61262

Cephadm task times out when waiting for osds to come up

Added by Laura Flores 11 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

/a/yuriw-2023-05-10_18:54:50-rados-wip-yuri-testing-2023-05-10-0917-distro-default-smithi/7270586

2023-05-11T15:27:46.051 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 11 15:27:45 smithi155 bash[25661]: audit 1683818865.0734086 mon.a (mon.0) 739 : audit 0 from='mgr.14152 172.21.15.77:0/1357883001' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch
2023-05-11T15:27:46.601 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":52,"num_osds":8,"num_up_osds":7,"osd_up_since":1683817624,"num_in_osds":7,"osd_in_since":1683818254,"num_remapped_pgs":0}
2023-05-11T15:27:46.602 ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/teuthology/contextutil.py", line 31, in nested
    vars.append(enter())
  File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "/home/teuthworker/src/github.com_ceph_ceph-c_40331d482836305f7c92885041944dbd6c79729f/qa/tasks/cephadm.py", line 888, in ceph_osds
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/teuthology/contextutil.py", line 135, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (120) after waiting for 120 seconds

This has happened previously in another test run:
/a/akupczyk-2023-04-06_15:26:20-rados-wip-aclamk-bs-esb-plus-legacy-distro-default-smithi/7234237

Also available in: Atom PDF