Project

General

Profile

Activity

From 04/01/2022 to 04/30/2022

04/30/2022

10:29 AM Bug #55258: lots of "heartbeat_check: no reply from X.X.X.X" in OSD logs
I did another run last night with 16 jobs on the kernel based on current mainline. Half of them failed, but I don't s... Jeff Layton

04/29/2022

10:11 PM Bug #55258: lots of "heartbeat_check: no reply from X.X.X.X" in OSD logs
The command I was using to test, per Venky's suggestion:... Jeff Layton
09:58 PM Bug #55258: lots of "heartbeat_check: no reply from X.X.X.X" in OSD logs
I tried to run a bisect today, but I think I ran afoul of some of the randomness in teuthology. Apparently even when ... Jeff Layton
12:20 PM Bug #55425: kclient: with 'wsync' option enabled with ffsb.sh will crash
... Xiubo Li
12:19 PM Bug #55425: kclient: with 'wsync' option enabled with ffsb.sh will crash
This seems not caused by kceph:... Xiubo Li
10:12 AM Bug #55090: mounting subvolume shows size/used bytes for entire fs, not subvolume
I think I managed to understand what's going on. It's a mix of kernel security features (LSMs) and cephfs authentica... Luis Henriques
10:00 AM Bug #55421: kclient: kernel BUG at fs/ceph/addr.c:125! when running the ffsb.sh
Xiubo Li wrote:
> It seems buggy in:
>
> [...]
>
> When doing writeback it will lock the pages and clear the d...
Xiubo Li
06:07 AM Bug #55421: kclient: kernel BUG at fs/ceph/addr.c:125! when running the ffsb.sh
It seems buggy in:... Xiubo Li

04/28/2022

12:49 PM Bug #55408 (Need More Info): libceph: corrupt inc osdmap (-12) epoch 409760 off 60 (ffffacad17925...
Dan Moraru wrote:
> A Scientific Linux 7.9 system running the latest kernel (3.10.0-1160.62.1.el7.x86_64) logged a "...
Ilya Dryomov
02:24 AM Bug #55421: kclient: kernel BUG at fs/ceph/addr.c:125! when running the ffsb.sh
Luis Henriques wrote:
> FWIW, I've seen this too with two different stack traces: the one in the previous comment an...
Xiubo Li

04/27/2022

04:06 PM Bug #55421: kclient: kernel BUG at fs/ceph/addr.c:125! when running the ffsb.sh
FWIW, I've seen this too with two different stack traces: the one in the previous comment and with this one:... Luis Henriques
11:55 AM Bug #55421: kclient: kernel BUG at fs/ceph/addr.c:125! when running the ffsb.sh
... Xiubo Li
11:55 AM Bug #55421: kclient: kernel BUG at fs/ceph/addr.c:125! when running the ffsb.sh
Xiubo Li wrote:
> The patchwork link: https://patchwork.kernel.org/project/ceph-devel/list/?series=635030
Except ...
Xiubo Li
02:41 PM Bug #55258: lots of "heartbeat_check: no reply from X.X.X.X" in OSD logs
One of the tests failed with a different softlockup with -rc4 based kernel:... Jeff Layton
12:07 PM Bug #55258: lots of "heartbeat_check: no reply from X.X.X.X" in OSD logs
I think this is probably unrelated to anything in the ceph patch pile. I see this in one the failed tests:... Jeff Layton
10:13 AM Bug #55258: lots of "heartbeat_check: no reply from X.X.X.X" in OSD logs
We're not carrying much in the testing kernel currently other than cephfs patches. We have a small libceph patch, but... Jeff Layton
09:04 AM Bug #55258: lots of "heartbeat_check: no reply from X.X.X.X" in OSD logs
Jeff,
I'vs schedule a couple of run - testing vs distro for fs:upgrade suite:
https://pulpito.ceph.com/vshankar...
Venky Shankar
04:15 AM Bug #55258: lots of "heartbeat_check: no reply from X.X.X.X" in OSD logs
Venky Shankar wrote:
> Neha Ojha wrote:
> > I am not seeing any daemon logs for vshankar-2022-04-09_12:55:41-fs-wip...
Venky Shankar

04/26/2022

04:22 PM Bug #55090: mounting subvolume shows size/used bytes for entire fs, not subvolume
Ok, I've managed to reproduce *a* bug with the same symptoms, but I suspect it's a different bug. Here's how I've do... Luis Henriques
03:16 PM Bug #55258: lots of "heartbeat_check: no reply from X.X.X.X" in OSD logs
Neha Ojha wrote:
> I am not seeing any daemon logs for vshankar-2022-04-09_12:55:41-fs-wip-vshankar-testing-55110-20...
Venky Shankar
06:31 AM Bug #46904 (In Progress): kclient: cluster [WRN] client.4478 isn't responding to mclientcaps(revoke)
Xiubo Li

04/25/2022

12:35 PM Bug #55377 (Fix Under Review): kclient: mds revoke Fwb caps stuck after the kclient tries writebc...
The patchwork link: https://patchwork.kernel.org/project/ceph-devel/list/?series=635295 Xiubo Li
10:41 AM Bug #55377: kclient: mds revoke Fwb caps stuck after the kclient tries writebcak once
That does look buggy. If there's no data then we don't need to write anything, but we do still need to update the inl... Jeff Layton
07:05 AM Bug #55377: kclient: mds revoke Fwb caps stuck after the kclient tries writebcak once
This bug is not introduce by the _*mmap()*_, it was introduce by the _*inline_data*_ and _*buffer write*_:
*1*, mo...
Xiubo Li
11:24 AM Bug #55425: kclient: with 'wsync' option enabled with ffsb.sh will crash
Jeff Layton wrote:
> Sounds like it actually just hung the box (or caused a lockup). Might need to set up kdump and ...
Xiubo Li
11:19 AM Bug #55425: kclient: with 'wsync' option enabled with ffsb.sh will crash
Sounds like it actually just hung the box (or caused a lockup). Might need to set up kdump and forcibly crash the box... Jeff Layton
10:38 AM Bug #55425 (Duplicate): kclient: with 'wsync' option enabled with ffsb.sh will crash
I have no crash core dump, when the kernel crash there has nothing in the terminal. It's very easy to reproduce if th... Xiubo Li
12:24 AM Bug #55421 (Fix Under Review): kclient: kernel BUG at fs/ceph/addr.c:125! when running the ffsb.sh
The patchwork link: https://patchwork.kernel.org/project/ceph-devel/list/?series=635030 Xiubo Li

04/24/2022

02:02 AM Bug #55421: kclient: kernel BUG at fs/ceph/addr.c:125! when running the ffsb.sh
The relevant code is:... Xiubo Li
02:00 AM Bug #55421 (Resolved): kclient: kernel BUG at fs/ceph/addr.c:125! when running the ffsb.sh
... Xiubo Li
12:51 AM Bug #54979 (Resolved): kclient: fs/ceph/mds_client.c:4476 check_session_state+0x55/0x60
Xiubo Li
12:49 AM Bug #55284 (Resolved): kclient: filesystem sync will stuck for around 5 seconds sometimes
Xiubo Li
12:47 AM Bug #53844 (Resolved): kclient: xfstest generic/003 failed with "access time has changed for file...
Have disabled the `atime` updating and we won't maintain it, updated the doc in https://github.com/ceph/ceph/pull/45979. Xiubo Li
12:44 AM Bug #55327 (Resolved): kclient: BUG: kernel NULL pointer dereference, address: 0000000000000008
Xiubo Li
12:39 AM Bug #55411 (Resolved): BUG: Dentry XXXXXX still in use (1) [unmount of ceph ceph]
Xiubo Li
12:08 AM Bug #55411: BUG: Dentry XXXXXX still in use (1) [unmount of ceph ceph]
Jeff Layton wrote:
> Found it, it's a recent regression in a patch that's in testing but not in mainline yet:
>
>...
Xiubo Li

04/22/2022

06:37 PM Bug #55258 (Need More Info): lots of "heartbeat_check: no reply from X.X.X.X" in OSD logs
I am not seeing any daemon logs for vshankar-2022-04-09_12:55:41-fs-wip-vshankar-testing-55110-20220408-203242-testin... Neha Ojha
05:39 PM Bug #55411 (Fix Under Review): BUG: Dentry XXXXXX still in use (1) [unmount of ceph ceph]
Found it, it's a recent regression in a patch that's in testing but not in mainline yet:
https://lore.kernel.o...
Jeff Layton
01:26 PM Bug #55411: BUG: Dentry XXXXXX still in use (1) [unmount of ceph ceph]
That looks unrelated at first glance and they may be false positives. Those allocations are all part of the mount cod... Jeff Layton
11:58 AM Bug #55411: BUG: Dentry XXXXXX still in use (1) [unmount of ceph ceph]
In the testing kernel I also could see this by running the ffsb and fsstress tests, not sure whether they are related... Xiubo Li
11:37 AM Bug #55411 (Resolved): BUG: Dentry XXXXXX still in use (1) [unmount of ceph ceph]
I've been able to fairly reliably reproduce a busy dentry at unmount problem with ceph by bouncing the MDS's regularl... Jeff Layton

04/21/2022

11:26 PM Bug #55408 (Need More Info): libceph: corrupt inc osdmap (-12) epoch 409760 off 60 (ffffacad17925...
A Scientific Linux 7.9 system running the latest kernel (3.10.0-1160.62.1.el7.x86_64) logged a "corrupt inc osdmap" m... Dan Moraru

04/20/2022

10:05 AM Bug #55090: mounting subvolume shows size/used bytes for entire fs, not subvolume
Jeff Layton wrote:
> I don't think a new kernel will probably help. The -365.el8 kernel is up to date with upstream ...
Luis Henriques
05:38 AM Bug #53844: kclient: xfstest generic/003 failed with "access time has changed for file in read-on...
Jeff Layton wrote:
> I'm not sure this is actually fixable. The test seems to require that the atime not change on a...
Xiubo Li
12:13 AM Bug #55377: kclient: mds revoke Fwb caps stuck after the kclient tries writebcak once
Jeff Layton wrote:
> The handling of the i_wrbuffer_ref value in the kernel seems broken by design. We bump that cou...
Xiubo Li

04/19/2022

05:42 PM Bug #55090: mounting subvolume shows size/used bytes for entire fs, not subvolume
I don't think a new kernel will probably help. The -365.el8 kernel is up to date with upstream as on ~December 2021. ... Jeff Layton
04:10 PM Bug #55090: mounting subvolume shows size/used bytes for entire fs, not subvolume
Dan van der Ster wrote:
> kernel is 4.18.0-365.el8.x86_64
Dan, can you please check with kernel >= 5.2 as pointed...
Ramana Raja
04:06 PM Bug #55090: mounting subvolume shows size/used bytes for entire fs, not subvolume
Ramana Raja wrote:
> Looks like this issue was fixed in kernel 5.2 https://tracker.ceph.com/issues/38482 ?
Yeah, ...
Luis Henriques
02:34 PM Bug #55090: mounting subvolume shows size/used bytes for entire fs, not subvolume
Looks like this issue was fixed in kernel 5.2 https://tracker.ceph.com/issues/38482 ? Ramana Raja
12:33 PM Bug #55377: kclient: mds revoke Fwb caps stuck after the kclient tries writebcak once
The handling of the i_wrbuffer_ref value in the kernel seems broken by design. We bump that counter whenever a page i... Jeff Layton
11:28 AM Bug #55377: kclient: mds revoke Fwb caps stuck after the kclient tries writebcak once
More detail please see https://tracker.ceph.com/issues/55240#note-4:
Another issue in this failure:
In _*mds.1*...
Xiubo Li
11:27 AM Bug #55377 (Resolved): kclient: mds revoke Fwb caps stuck after the kclient tries writebcak once
Seen here: https://pulpito.ceph.com/vshankar-2022-04-07_05:07:33-fs-master-testing-default-smithi/6780578/
Its an ...
Xiubo Li

04/18/2022

01:58 AM Bug #55327: kclient: BUG: kernel NULL pointer dereference, address: 0000000000000008
Xiubo Li wrote:
> The patchwork link is https://patchwork.kernel.org/project/ceph-devel/list/?series=632113
The V...
Xiubo Li

04/14/2022

05:51 AM Bug #55327: kclient: BUG: kernel NULL pointer dereference, address: 0000000000000008
I added on test case for this issue in https://tracker.ceph.com/issues/55329. Xiubo Li
05:50 AM Bug #55327 (Fix Under Review): kclient: BUG: kernel NULL pointer dereference, address: 0000000000...
The patchwork link is https://patchwork.kernel.org/project/ceph-devel/list/?series=632113 Xiubo Li
05:32 AM Bug #55327: kclient: BUG: kernel NULL pointer dereference, address: 0000000000000008
The bugzilla link https://bugzilla.redhat.com/show_bug.cgi?id=2075068 Xiubo Li
01:11 AM Bug #55327 (In Progress): kclient: BUG: kernel NULL pointer dereference, address: 0000000000000008
Xiubo Li
01:10 AM Bug #55327 (Resolved): kclient: BUG: kernel NULL pointer dereference, address: 0000000000000008
More detail please see https://tracker.ceph.com/issues/53809#note-8. Xiubo Li

04/13/2022

02:52 PM Bug #55052: when mounting with new dev syntax and -o ms_mode=legacy, the option is not shown
We probably want to move towards the mount helper *not* pass in ms_mode to the kernel (if one isn't specified) and le... Venky Shankar

04/12/2022

12:38 PM Bug #55284 (Fix Under Review): kclient: filesystem sync will stuck for around 5 seconds sometimes
The patchwork link: https://patchwork.kernel.org/project/ceph-devel/list/?series=631476 Xiubo Li
09:05 AM Bug #55284 (In Progress): kclient: filesystem sync will stuck for around 5 seconds sometimes
Xiubo Li
09:04 AM Bug #55284 (Resolved): kclient: filesystem sync will stuck for around 5 seconds sometimes
We have fixed the fsync(fd) case in the following commits:... Xiubo Li

04/11/2022

01:52 PM Bug #55254 (Resolved): kclient: fix statx AT_STATX_DONT_SYNC vs AT_STATX_FORCE_SYNC check
Xiubo Li
03:04 AM Bug #55254 (Fix Under Review): kclient: fix statx AT_STATX_DONT_SYNC vs AT_STATX_FORCE_SYNC check
Xiubo Li
03:01 AM Bug #55254: kclient: fix statx AT_STATX_DONT_SYNC vs AT_STATX_FORCE_SYNC check
The ceph patchwork link: https://patchwork.kernel.org/project/ceph-devel/list/?series=630867 Xiubo Li
03:00 AM Bug #55254 (Resolved): kclient: fix statx AT_STATX_DONT_SYNC vs AT_STATX_FORCE_SYNC check
From the posix and the initial statx supporting commit comments,
the AT_STATX_DONT_SYNC is a lightweight stat flag a...
Xiubo Li
12:19 PM Bug #55258 (Resolved): lots of "heartbeat_check: no reply from X.X.X.X" in OSD logs
Seeing this in upgrade suite for CephFS and seems to be happening frequently: https://pulpito.ceph.com/vshankar-2022-... Venky Shankar
 

Also available in: Atom