Project

General

Profile

Bug #20569

mds: don't mark dirty rstat on non-auth inode

Added by Zhi Zhang over 6 years ago. Updated about 5 years ago.

Status:
Resolved
Priority:
Normal
Category:
-
Target version:
-
% Done:

0%

Source:
Community (dev)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Labels (FS):
multimds
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Currently using multi-MDS on Luminous, we found ceph status reported such warning all the time if writing large amount of files (e.g, 1 million) into a directory and migration had happened from MDS.0 to MDS.1

cluster 63dd6fd2-78d5-49b0-9b3f-5b69fcbd4b25
     health HEALTH_WARN
            mds1: Too many inodes in cache (602815/100000), 92628 inodes in use by clients, 0 stray files
            noscrub,nodeep-scrub flag(s) set
     monmap e3: 1 mons at {c167=xxx:6789/0}
            election epoch 38, quorum 0 c167
      fsmap e10115: 2/2/2 up {0=c166=up:active,1=c167=up:active}, 1 up:standby-replay

From MDS perf dump, we can see inodes with caps were less than 100000, but total inodes in cache were too many.

[ceph@c167 ~]$ sudo ceph --admin-daemon /var/run/ceph/ceph-mds.c167.asok perf dump | grep inode
        "inode_max": 100000,
        "inodes": 602815,
        "inodes_top": 0,
        "inodes_bottom": 0,
        "inodes_pin_tail": 602815,
        "inodes_pinned": 602815,
        "inodes_expired": 706862,
        "inodes_with_caps": 92628,
        "exported_inodes": 0,
        "imported_inodes": 113536

MDS trim job couldn't trim those inodes because they were not expired in LRU. From MDS cache dump, we can see those inodes' "dirtyrstat" was still 1 but "caps" or other flags were already 0.

[inode 20001010c3e [2,head] /1499743036981_0/file_895909 auth v130471 s=73728 n(v0 b73728 1=1+0)/n(v0 1=1+0) (iversion lock) | ptrwaiter=0 request=0 lock=0 caps=0 dirtyrstat=1 dirtyparent=0 dirty=0 authpin=0 0x7f1cd7e5ca00]
[inode 20001010c3d [2,head] /1499743036981_0/file_895908 auth v137783 s=73728 n(v0 b73728 1=1+0)/n(v0 1=1+0) (iversion lock) | ptrwaiter=0 request=0 lock=0 caps=0 dirtyrstat=1 dirtyparent=0 dirty=0 authpin=0 0x7f1cd7e5d000]
...

The reason is that those inodes had been exported to MDS.1 and marked with dirtyrstat in predirty_journal_parents, but predirty_journal_parents was stopped later if this inode's parent was not auth for MDS.1. So its dirtyrstat flag won't been cleared.

I think we don't need to mark dirtyrstat on this inode on non-auth MDS. Auth MDS will do such things on this inode's parent.

History

#1 Updated by Zhi Zhang over 6 years ago

https://github.com/ceph/ceph/pull/16253

Within this fix, inodes can be trimmed successfully and no such warning reported on replicated MDS.

#3 Updated by Patrick Donnelly over 6 years ago

  • Status changed from New to 7

#4 Updated by Patrick Donnelly over 6 years ago

  • Assignee set to Patrick Donnelly

#5 Updated by Patrick Donnelly over 6 years ago

  • Status changed from 7 to Resolved

#6 Updated by Patrick Donnelly about 5 years ago

  • Category deleted (90)
  • Labels (FS) multimds added

Also available in: Atom PDF