Bug #37278: filefrag receives SIGFPE when run on any file on CephFS (kernel mount) - Linux kernel client - Ceph

Actions

Copy link

Bug #37278

closed

filefrag receives SIGFPE when run on any file on CephFS (kernel mount)

Added by Марк Коренберг over 5 years ago. Updated about 5 years ago.

Status:

Resolved

Priority:

Normal

Assignee:

Zheng Yan

Category:

fs/ceph

Target version:

% Done:

Source:

Community (user)

Tags:

Backport:

Regression:

Severity:

4 - irritation

Reviewed:

Affected Versions:

ceph-qa-suite:

Crash signature (v1):

Crash signature (v2):

Description

Such hilarious thing happens because fstat() on any file returns .st_dev=0. filefrag tracks that as uninitialized value in its internal logic crashes.

For example, /proc files give st_dev=makedev(0, 21), /sys files give st_dev=makedev(0, 20).

I think, CephFS should give st_dev=makedev(0, <something>).

(and also, it would be nice to support FIBMAP ioctls and so on, but this is a different task)

Actions

Copy link

Updated by Patrick Donnelly over 5 years ago

Assignee set to Zheng Yan
Target version deleted (~~v12.2.9~~)
Affected Versions deleted (~~v12.2.7, v12.2.8, v13.0.0, v13.2.0, v13.2.1, v13.2.2, v14.0.0~~)
ceph-qa-suite deleted (~~fs, kcephfs~~)

Actions

Copy link

Updated by Zheng Yan over 5 years ago

ceph_getattr() does set set_dev. which kernel do you use.

Actions

Copy link

Updated by Марк Коренберг over 5 years ago

uname -a
Linux mmwork 4.18.0-11-generic #12~18.04.1-Ubuntu SMP Thu Oct 25 13:04:42 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Actions

Copy link

Updated by Марк Коренберг over 5 years ago

strace:

fstat(3, {st_dev=makedev(0, 0), st_ino=1099513576238, st_mode=S_IFREG|0777, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4194304, st_blocks=2, st_size=608, st_atime=1542342442 /* 2018-11-16T09:27:22.833125000+0500 */, st_atime_nsec=833125000, st_mtime=1542342467 /* 2018-11-16T09:27:47.841310000+0500 */, st_mtime_nsec=841310000, st_ctime=1542342467 /* 2018-11-16T09:27:47.841310000+0500 */, st_ctime_nsec=841310000}) = 0
--- SIGFPE {si_signo=SIGFPE, si_code=FPE_INTDIV, si_addr=0x55842e451145} ---
+++ killed by SIGFPE +++

Actions

Copy link

Updated by Zheng Yan over 5 years ago

Is it snapshot inode?

Actions

Copy link

Updated by Марк Коренберг over 5 years ago

No, it is just plain file. I have never used snapshots on this CephFS.

Actions

Copy link

Updated by Zheng Yan over 5 years ago

Status changed from New to 7

commit "ceph: map snapid to anonymous bdev ID" in testing branch

https://github.com/ceph/ceph-client/commit/f99f6c5ff02f3020e448d125eb339d645f27ffe9

Actions

Copy link

Updated by Марк Коренберг over 5 years ago

Finally, I have checked different stat fields on same fs mounted through kernel and through FUSE:

(I removed fields with exactly same values)

ftp is a kernel-mounted FS
ftp2 is a FUSE-mounted FS

# strace -v -e statfs stat -f ftp/qwe >/dev/null
statfs("ftp/qwe", {
f_files=784708,                        // DIFFERS!
f_ffree=18446744073709551615,          // DIFFERS! very BIG
f_fsid={val=[3300624616, 1863187541]}, // DIFFERS, here is good one.
}) = 0

# strace -v -e statfs stat -f ftp2/qwe >/dev/null
statfs("ftp2/qwe", {
f_files=4203,        // DIFFERS!
f_ffree=0,           // ZERO ?!
f_fsid={val=[0, 0]}, // ZERO !!!
}) = 0

# strace -e lstat -v stat ftp/qwe >/dev/null
lstat("ftp/qwe", {
st_dev=makedev(0, 0),    // DIFFERS! ZEROES!
}) = 0

# strace -e lstat -v stat ftp2/qwe >/dev/null
lstat("ftp2/qwe", {
st_dev=makedev(0, 43),  // DIFFERS. seems good one.
}) = 0

Actions

Copy link

Updated by Марк Коренберг over 5 years ago

Did not say,

f_files=784708, IS WRONG
f_files=4203 IS CORRECT.

Note, I mounted subtree of big fs.

Actions

Copy link

#10

Updated by Марк Коренберг over 5 years ago

It seems, there are many bugs connected with stat/fstat. Please guide me how to separate this issue to different ones if it is required.

Actions

Copy link

#11

Updated by Zheng Yan over 5 years ago

yes, statfs are inconsistent between ceph-fuse and kclient. check ceph_statfs() in kernel-src/fs/ceph/super.c and Client::statfs() in ceph-src/src/client/Client.cc

Actions

Copy link

#12