Bug #48411
tasks.cephfs.test_volumes.TestSubvolumeGroups: RuntimeError: rank all failed to reach desired subtree state
% Done:
0%
Source:
Q/A
Tags:
Backport:
pacific,octopus,nautilus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
MDS, qa-suite
Labels (FS):
qa-failure
Pull request ID:
Crash signature:
Description
I got this failure when doing some testing with the draft fscache rework. It looks unrelated to the kernel changes, and more like a bug in the MDS:
2020-12-01T14:17:03.365 INFO:tasks.cephfs_test_runner:test_subvolumegroup_pin_distributed (tasks.cephfs.test_volumes.TestSubvolumeGroups) ... ERROR 2020-12-01T14:17:03.366 INFO:tasks.cephfs_test_runner: 2020-12-01T14:17:03.367 INFO:tasks.cephfs_test_runner:====================================================================== 2020-12-01T14:17:03.368 INFO:tasks.cephfs_test_runner:ERROR: test_subvolumegroup_pin_distributed (tasks.cephfs.test_volumes.TestSubvolumeGroups) 2020-12-01T14:17:03.368 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2020-12-01T14:17:03.369 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2020-12-01T14:17:03.369 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_jtlayton_ceph_k-stock/qa/tasks/cephfs/cephfs_test_case.py", line 380, in _wait_distributed_subtrees 2020-12-01T14:17:03.369 INFO:tasks.cephfs_test_runner: while proceed(): 2020-12-01T14:17:03.370 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 133, in __call__ 2020-12-01T14:17:03.370 INFO:tasks.cephfs_test_runner: raise MaxWhileTries(error_msg) 2020-12-01T14:17:03.370 INFO:tasks.cephfs_test_runner:teuthology.exceptions.MaxWhileTries: reached maximum tries (20) after waiting for 100 seconds 2020-12-01T14:17:03.371 INFO:tasks.cephfs_test_runner: 2020-12-01T14:17:03.371 INFO:tasks.cephfs_test_runner:The above exception was the direct cause of the following exception: 2020-12-01T14:17:03.371 INFO:tasks.cephfs_test_runner: 2020-12-01T14:17:03.371 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2020-12-01T14:17:03.372 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_jtlayton_ceph_k-stock/qa/tasks/cephfs/test_volumes.py", line 694, in test_subvolumegroup_pin_distributed 2020-12-01T14:17:03.372 INFO:tasks.cephfs_test_runner: self._wait_distributed_subtrees(2 * 2, status=status, rank="all") 2020-12-01T14:17:03.373 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_jtlayton_ceph_k-stock/qa/tasks/cephfs/cephfs_test_case.py", line 389, in _wait_distributed_subtrees 2020-12-01T14:17:03.374 INFO:tasks.cephfs_test_runner: raise RuntimeError("rank {0} failed to reach desired subtree state".format(rank)) from e 2020-12-01T14:17:03.374 INFO:tasks.cephfs_test_runner:RuntimeError: rank all failed to reach desired subtree state 2020-12-01T14:17:03.375 INFO:tasks.cephfs_test_runner: 2020-12-01T14:17:03.375 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2020-12-01T14:17:03.375 INFO:tasks.cephfs_test_runner:Ran 17 tests in 581.397s 2020-12-01T14:17:03.376 INFO:tasks.cephfs_test_runner: 2020-12-01T14:17:03.376 INFO:tasks.cephfs_test_runner:FAILED (errors=1)
History
#1 Updated by Patrick Donnelly about 2 months ago
- Status changed from New to Triaged
- Assignee set to Patrick Donnelly
- Priority changed from Normal to High
- Target version set to v16.0.0
- Source set to Q/A
- Component(FS) MDS, qa-suite added
- Labels (FS) qa-failure added
#2 Updated by Patrick Donnelly 6 days ago
- Target version changed from v16.0.0 to v17.0.0
- Backport set to pacific,octopus,nautilus