Bug #24849
closedclient: statfs inode count odd
0%
Description
Hi,
We noticed that df -i on our cehpfs was a bit odd:
Filesystem Inodes IUsed IFree IUse% Mounted on 10.xx.xx.xx:6789,10.xx.xx.xx:6789,10.xx.xx.xx:6789:/svp/ 10984832 - - - /mnt/cephfs
Stracing I find this:
statfs("/mnt/cephfs", {f_type=0xc36400, f_bsize=4194304, f_blocks=15611049, f_bfree=7289822, f_bavail=7289822, f_files=10979519, f_ffree=18446744073709551615, f_fsid={1894312162, 282397230}, f_namelen=255, f_frsize=4194304, f_flags=ST_VALID|ST_NOATIME}) = 0
man page shows
fsfilcnt_t f_files; /* Total file nodes in filesystem */
fsfilcnt_t f_ffree; /* Free file nodes in filesystem */
From the source code it seems clear that f_files is the total number of file nodes in the file system, used or not so it's a bit startling that the number of free nodes is higher than the number of nodes in total.
The nagios checker shows the filesystem to have -.0 free inodes due to quirks of math.
Client kernel is stock centos 7:
Linux svp-encoder-03.int.vgnett.no 3.10.0-862.3.3.el7.x86_64 #1 SMP Fri Jun 15 04:15:27 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
On the server side we run "luminous" on stock CentOS 7. The package versions are 12.2.5-0.el7.x86_64
Updated by John Spray almost 6 years ago
We set f_ffree to -1 because it's not meaningful (unlike e.g. ext4, we don't really have an inode limit other than the range of 64 bit integers), and we set f_files to the number of objects because... well actually that's kind of a weird thing to do. I don't know why we aren't setting f_files to the number of files in the filesystem, which we could get from the rstats on the root.
Updated by Patrick Donnelly almost 6 years ago
- Subject changed from statfs inode count odd to client: statfs inode count odd
- Assignee set to Rishabh Dave
- Priority changed from Normal to High
- Target version set to v14.0.0
- Source set to Community (user)
- Backport set to mimic,luminous
- Component(FS) Client, kceph added
- Labels (FS) task(easy) added
Updated by Nicolai Langfeldt almost 6 years ago
How about setting
f_files = -1
and then
f_ffree = f_files - number_of_objects?
... and then hope you don't get more than -1 objects in the store? That should be more along the lines of what clients expect?
Updated by Rishabh Dave over 5 years ago
Assigns total number of files on FS (directories are counted as files) to f_files - https://github.com/ceph/ceph/pull/23323
Updated by Rishabh Dave over 5 years ago
- Status changed from New to Fix Under Review
Updated by Patrick Donnelly over 5 years ago
- Status changed from Fix Under Review to Pending Backport
Updated by Nathan Cutler over 5 years ago
- Copied to Backport #35939: luminous: client: statfs inode count odd added
Updated by Nathan Cutler over 5 years ago
- Copied to Backport #35940: mimic: client: statfs inode count odd added
Updated by Nathan Cutler over 5 years ago
- Status changed from Pending Backport to Resolved