Project

General

Profile

Actions

Bug #41728

closed

mds: hang during fragmentdir

Added by Nathan Fish over 4 years ago. Updated about 4 years ago.

Status:
Can't reproduce
Priority:
High
Assignee:
Category:
Correctness/Safety
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
Backport:
nautilus,mimic
Regression:
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
MDS, osdc
Labels (FS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

When doing a parallel cp, the active MDS on the CephFS hung on a fragmentdir op.
It might be this bug: http://lists.ceph.com/pipermail/ceph-users-ceph.com/2019-May/034785.html

I was able to repair the cluster by failing the active MDS, unmounting and remounting the client, then failing the new active MDS.
I increased "mds bal split rd" to 250000 and "mds bal split wr" to 100000 in the hopes this would reduce the number of fragmentdir ops,
then restarted the cp. The issue occurred again an hour or two in.

Active MDS ops:
ceph daemon mds.dc-3558-422-C ops
https://termbin.com/c3sm

ceph health detail
https://termbin.com/xx4k
All machines are Ubuntu 18.04 with hwe kernel 5.0, and Nautilus 14.2.3.


Related issues 1 (0 open1 closed)

Related to CephFS - Bug #41434: mds: infinite loop in Locker::file_update_finish()ResolvedZheng Yan

Actions
Actions #1

Updated by Patrick Donnelly over 4 years ago

  • Subject changed from MDS hangs during fragmentdir to mds: hang during fragmentdir
  • Priority changed from Normal to High
  • Target version set to v15.0.0
  • Start date deleted (09/09/2019)
  • Backport set to nautilus,mimic
  • Component(FS) MDS added
Actions #2

Updated by Zheng Yan over 4 years ago

  • Component(FS) osdc added

Nathan Fish wrote:

When doing a parallel cp, the active MDS on the CephFS hung on a fragmentdir op.
It might be this bug: http://lists.ceph.com/pipermail/ceph-users-ceph.com/2019-May/034785.html

Above bug is fixed by https://github.com/ceph/ceph/pull/29902

Actions #3

Updated by Nathan Fish over 4 years ago

Thanks!

Actions #4

Updated by Patrick Donnelly over 4 years ago

  • Related to Bug #41434: mds: infinite loop in Locker::file_update_finish() added
Actions #5

Updated by Patrick Donnelly over 4 years ago

  • Status changed from New to Need More Info
  • Assignee set to Zheng Yan
Actions #6

Updated by Patrick Donnelly over 4 years ago

  • Target version deleted (v15.0.0)
Actions #7

Updated by Zheng Yan about 4 years ago

  • Status changed from Need More Info to Can't reproduce
Actions

Also available in: Atom PDF