Bug #46732: teuthology.exceptions.MaxWhileTries: 'check for active or peered' reached maximum tries (5) after waiting for 25 seconds seen on octopus - RADOS - Ceph

Actions

Copy link

Bug #46732

open

teuthology.exceptions.MaxWhileTries: 'check for active or peered' reached maximum tries (5) after waiting for 25 seconds seen on octopus

Added by Brad Hubbard over 3 years ago. Updated over 3 years ago.

Status:

Need More Info

Priority:

Normal

Assignee:

Category:

Target version:

% Done:

Source:

Tags:

Backport:

octopus

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Component(RADOS):

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

/a/yuriw-2020-07-13_23:00:15-rados-wip-yuri8-testing-2020-07-13-1946-octopus-distro-basic-smithi/5223971

2020-07-14T11:46:41.479 INFO:tasks.thrashosds.thrasher:not all PGs are active or peered
2020-07-14T11:46:41.480 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last):
  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-yuri8-testing-2020-07-13-1946-octopus/qa/tasks/ceph_manager.py", line 122, in wrapper
    return func(self)
  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-yuri8-testing-2020-07-13-1946-octopus/qa/tasks/ceph_manager.py", line 1208, in _do_thrash
    self.choose_action()()
  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-yuri8-testing-2020-07-13-1946-octopus/qa/tasks/ceph_manager.py", line 880, in test_pool_min_size
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'check for active or peered' reached maximum tries (5) after waiting for 25 seconds

Actions

Copy link

Updated by Brad Hubbard over 3 years ago

Description updated (diff)

Actions

Copy link

Updated by Brad Hubbard over 3 years ago

Status changed from New to Need More Info

Looks like osd.2 was taken down by the thrasher and did not come back up. We'd probably need a full set of logs to work out why.

Actions

Copy link

Updated by Neha Ojha over 3 years ago

2020-07-29T20:11:52.883 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-35628-2020-07-28/qa/tasks/ceph_manager.py", line 118, in wrapper
    return func(self)
  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-35628-2020-07-28/qa/tasks/ceph_manager.py", line 1204, in _do_thrash
    self.choose_action()()
  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-35628-2020-07-28/qa/tasks/ceph_manager.py", line 876, in test_pool_min_size
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'check for active or peered' reached maximum tries (5) after waiting for 25 seconds

rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/fastclose rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-fast-read-overwrites}

/a/nojha-2020-07-29_19:10:54-rados-wip-35628-2020-07-28-distro-basic-smithi/5267458 - branch based on master

Actions

Copy link