Project

General

Profile

Bug #24849

client: statfs inode count odd

Added by Nicolai Langfeldt 5 months ago. Updated about 2 months ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
-
Target version:
Start date:
07/10/2018
Due date:
% Done:

0%

Source:
Community (user)
Tags:
Backport:
mimic,luminous
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Client, kceph
Labels (FS):
task(easy)
Pull request ID:

Description

Hi,

We noticed that df -i on our cehpfs was a bit odd:

Filesystem                                                 Inodes IUsed IFree IUse% Mounted on
10.xx.xx.xx:6789,10.xx.xx.xx:6789,10.xx.xx.xx:6789:/svp/ 10984832     -     -     - /mnt/cephfs

Stracing I find this:

statfs("/mnt/cephfs", {f_type=0xc36400, f_bsize=4194304, f_blocks=15611049, f_bfree=7289822, f_bavail=7289822, f_files=10979519, f_ffree=18446744073709551615, f_fsid={1894312162, 282397230}, f_namelen=255, f_frsize=4194304, f_flags=ST_VALID|ST_NOATIME}) = 0

man page shows

           fsfilcnt_t f_files;   /* Total file nodes in filesystem */
           fsfilcnt_t f_ffree;   /* Free file nodes in filesystem */

From the source code it seems clear that f_files is the total number of file nodes in the file system, used or not so it's a bit startling that the number of free nodes is higher than the number of nodes in total.

The nagios checker shows the filesystem to have -.0 free inodes due to quirks of math.

Client kernel is stock centos 7:

Linux svp-encoder-03.int.vgnett.no 3.10.0-862.3.3.el7.x86_64 #1 SMP Fri Jun 15 04:15:27 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

On the server side we run "luminous" on stock CentOS 7. The package versions are 12.2.5-0.el7.x86_64


Related issues

Copied to fs - Backport #35939: luminous: client: statfs inode count odd Resolved
Copied to fs - Backport #35940: mimic: client: statfs inode count odd Resolved

History

#1 Updated by John Spray 5 months ago

We set f_ffree to -1 because it's not meaningful (unlike e.g. ext4, we don't really have an inode limit other than the range of 64 bit integers), and we set f_files to the number of objects because... well actually that's kind of a weird thing to do. I don't know why we aren't setting f_files to the number of files in the filesystem, which we could get from the rstats on the root.

#2 Updated by John Spray 5 months ago

  • Project changed from Ceph to fs

#3 Updated by Patrick Donnelly 5 months ago

  • Subject changed from statfs inode count odd to client: statfs inode count odd
  • Assignee set to Rishabh Dave
  • Priority changed from Normal to High
  • Target version set to v14.0.0
  • Source set to Community (user)
  • Backport set to mimic,luminous
  • Component(FS) Client, kceph added
  • Labels (FS) task(easy) added

#4 Updated by Nicolai Langfeldt 5 months ago

How about setting

  f_files = -1

and then

  f_ffree = f_files - number_of_objects? 

... and then hope you don't get more than -1 objects in the store? That should be more along the lines of what clients expect?

#5 Updated by Rishabh Dave 5 months ago

Assigns total number of files on FS (directories are counted as files) to f_files - https://github.com/ceph/ceph/pull/23323

#6 Updated by Rishabh Dave 5 months ago

  • Status changed from New to Need Review

#7 Updated by Patrick Donnelly 3 months ago

  • Status changed from Need Review to Pending Backport

#8 Updated by Nathan Cutler 3 months ago

  • Copied to Backport #35939: luminous: client: statfs inode count odd added

#9 Updated by Nathan Cutler 3 months ago

#10 Updated by Nathan Cutler about 2 months ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF