Project

General

Profile

Actions
Copy link

Bug #37716

open

failed to recover before timeout expired due to pgs going into backfill_toofull

Added by Neha Ojha over 5 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2.1d          0                  0        0         0       0        0    0        0                                     active+clean 2018-12-19 04:36:09.178688      0'0 939:1939   [7,6,9,3,0,4]          7   [7,6,9,3,0,4]              7        0'0 2018-12-19 04:35:36.853437             0'0 2018-12-19 04:32:49.953703             0
2.1c         63                  0        0       378       0 98680832  279      279                 active+remapped+backfill_toofull 2018-12-19 04:35:56.811461  940'351 940:2181  [7,11,5,4,8,6]          7   [6,5,0,1,7,9]              6    173'123 2018-12-19 04:14:56.971177             0'0 2018-12-19 04:12:24.157274           230
2.1b          8                  0        0         0       0 18595840  660      660                                     active+clean 2018-12-19 04:36:09.206537  939'736 939:3277  [5,6,11,7,9,3]          5  [5,6,11,7,9,3]              5    899'698 2018-12-19 04:35:26.627175         899'698 2018-12-19 04:35:26.627175             0
2.1a         21                  0        0       105       0 17408000  107      107 active+forced_recovery+remapped+backfill_toofull 2018-12-19 04:35:58.076595  935'146 939:1955  [11,4,7,2,1,5]         11   [6,3,7,1,5,2]              6     119'32 2018-12-19 04:14:07.290308             0'0 2018-12-19 04:12:24.157274           230

causing

2018-12-19T04:36:13.581 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last):
  File "/home/teuthworker/src/github.com_ceph_ceph_master/qa/tasks/ceph_manager.py", line 883, in wrapper
    return func(self)
  File "/home/teuthworker/src/github.com_ceph_ceph_master/qa/tasks/ceph_manager.py", line 1006, in do_thrash
    timeout=self.config.get('timeout')
  File "/home/teuthworker/src/github.com_ceph_ceph_master/qa/tasks/ceph_manager.py", line 2252, in wait_for_recovery
    'failed to recover before timeout expired'
AssertionError: failed to recover before timeout expired

/a/nojha-2018-12-19_01:41:09-rados-master-distro-basic-smithi/3375364/

No data to display

Actions
Copy link

Also available in: Atom PDF