Project

General

Profile

Bug #45835

mds: OpenFileTable::prefetch_inodes during rejoin can cause out-of-memory

Added by Dan van der Ster about 1 month ago. Updated about 1 month ago.

Status:
Triaged
Priority:
Normal
Assignee:
Category:
-
Target version:
% Done:

0%

Source:
Community (dev)
Tags:
Backport:
octopus,nautilus
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
fs
Component(FS):
MDS
Labels (FS):
Pull request ID:
Crash signature:

Description

We just upgraded from mimic v13.2.6 to nautilus v14.2.9 and the single active MDS was going out-of-memory during the rejoin step.
MDS cache size was 4GB, but the OOM occured on 16GB and even 32GB VMs.

`mds_wipe_sessions = true` did not help.

To recover the cluster I did `rados -p cephfs_metadata_pool rm mds0_openfiles.0` then the MDS could activate quickly.

History

#1 Updated by Patrick Donnelly about 1 month ago

  • Subject changed from OpenFileTable::prefetch_inodes during rejoin can cause out-of-memory to mds: OpenFileTable::prefetch_inodes during rejoin can cause out-of-memory
  • Status changed from New to Triaged
  • Assignee set to Zheng Yan
  • Target version set to v16.0.0
  • Source set to Community (dev)
  • Backport set to octopus,nautilus

Also available in: Atom PDF