Activity
From 07/13/2015 to 08/11/2015
08/11/2015
- 09:51 PM Feature #12671: Enforce cache limit during dirfrag load during open_ino (during rejoin)
- The source of this observation was https://www.mail-archive.com/ceph-users@lists.ceph.com/msg22235.html
In this in... - 09:49 PM Feature #12671 (New): Enforce cache limit during dirfrag load during open_ino (during rejoin)
When clients replay requests referring to inodes not found in cache, the inode numbers are stashed for loading late...- 02:01 PM Bug #12617: Missing handling of supplementary groups in ceph-fuse -> rendering it useless for sha...
- Do we need the Client.cc patch here, or just the pull request (https://github.com/ceph/ceph/pull/5480), or both?
- 02:00 PM Bug #12321: MDS crash when try to connect clients
- Hi, do you have any updates for us on this? If the system is unavailable for any more debug then we'll close this.
- 01:55 PM Bug #12619 (Resolved): valgrind errors in fs suite
- 01:32 PM Bug #11482: kclient: intermittent log warnings "client.XXXX isn't responding to mclientcaps(revoke)"
- We need to turn on MDS logs to get a more useful bit of information
- 09:07 AM Bug #12209: CephFS should have a complete timeout mechanism to avoid endless waiting or unpredict...
- This issue is covered in http://tracker.ceph.com/issues/10944, and we have tested the two patches, which works well. ...
08/10/2015
- 09:07 PM Bug #11482: kclient: intermittent log warnings "client.XXXX isn't responding to mclientcaps(revoke)"
- Reopening this because we're still seeing it on kcephfs, unknown whether underlying cause is in the same area:
http:... - 06:40 PM Bug #11255: nfs: mount failures on ceph-backed NFS share
- http://pulpito.ceph.com/teuthology-2015-08-07_23:10:02-knfs-master-testing-basic-multi/1006003/
- 06:31 PM Bug #12657: Failure in TestStrays.test_ops_throttle
- This purged 1600 strays in 6 seconds (and therefore wasn't busy doing so for long enough to hit the required high wat...
- 06:30 PM Bug #12657 (Can't reproduce): Failure in TestStrays.test_ops_throttle
http://pulpito.ceph.com/teuthology-2015-08-07_23:04:03-fs-master---basic-multi/1005757/- 08:03 AM Bug #12653: fuse mounted file systems fails SAMBA CTDB ping_pong rw test with v9.0.2
- This is a ceph-fuse bug, running the test directly on ceph-fuse mount can reproduce the bug. The bug is because that ...
- 03:37 AM Bug #12653 (Resolved): fuse mounted file systems fails SAMBA CTDB ping_pong rw test with v9.0.2
- While working with the SAMBA CTDB ping_pong test and the Ceph file system, I noticed that file systems mounted using ...
08/07/2015
- 08:42 AM Bug #10944: Deadlock, MDS logs "slow request", getattr pAsLsXsFs failed to rdlock
- please provide backtrace of the crash
- 07:33 AM Bug #10944: Deadlock, MDS logs "slow request", getattr pAsLsXsFs failed to rdlock
- Zheng Yan wrote:
> will do
HI,i use your patch in ceph dokan client. and seems cause to client assert crash.
D... - 02:26 AM Bug #12189 (Won't Fix): Editing / Creating files fails for NFS-over-CephFS on EC pool with cache ...
08/06/2015
- 04:25 PM Bug #12189: Editing / Creating files fails for NFS-over-CephFS on EC pool with cache tier
- I was able to update the kernel on the NFS server to version 4.1.4 today, which also allows me to use the kernel clie...
- 11:16 AM Bug #12619: valgrind errors in fs suite
- 08:15 AM Bug #12619 (Fix Under Review): valgrind errors in fs suite
- https://github.com/ceph/ceph/pull/5487
- 09:14 AM Bug #12617: Missing handling of supplementary groups in ceph-fuse -> rendering it useless for sha...
- Burkhard Linke wrote:
> The following patch adds correct supplementary group retrieval:
>
> If no callback is def...
08/05/2015
- 02:29 PM Bug #12617: Missing handling of supplementary groups in ceph-fuse -> rendering it useless for sha...
- see #3301, and commit:328bf482349485b90911fe8953d8ce5f0769c69e for when this was introduced... hopefully we aren't re...
- 02:27 PM Bug #12617: Missing handling of supplementary groups in ceph-fuse -> rendering it useless for sha...
- The following patch adds correct supplementary group retrieval:
If no callback is defined, the getgrouplist() func... - 02:25 PM Bug #12617 (Fix Under Review): Missing handling of supplementary groups in ceph-fuse -> rendering...
- https://github.com/ceph/ceph/pull/5480
- 11:30 AM Bug #12617: Missing handling of supplementary groups in ceph-fuse -> rendering it useless for sha...
- I checked the kernel code, It seem that kernel also does the permission check. can we just remove the permission in c...
- 08:06 AM Bug #12617 (Resolved): Missing handling of supplementary groups in ceph-fuse -> rendering it usel...
- During investigating a user's bug report I've stumbled accross
https://github.com/ceph/ceph/blob/master/src/client... - 12:35 PM Bug #12619 (Resolved): valgrind errors in fs suite
- Unclear if these are really CephFS bugs (the SyscallParam thing especially), but it's our tests they're coming up in....
- 11:32 AM Bug #12578 (Resolved): multimds: mds/SnapRealm.cc: 238: FAILED assert(open)
- 10:54 AM Bug #12612: fuse jobs fail to start on centos7
- Have these centos7 nodes run other tests successfully? There's a bunch of spurious failures from missing centos7 pac...
- 10:14 AM Bug #12612 (Need More Info): fuse jobs fail to start on centos7
- ceph-fuse fails with -ETIMEOUT, no information why timeout happened. In this test, fuse_mount.py kept checking /sys/f...
- 07:50 AM Bug #12598 (Fix Under Review): LibCephFS.GetPoolId failure
- https://github.com/ceph/ceph/pull/5477
08/04/2015
- 09:49 PM Bug #12612 (Can't reproduce): fuse jobs fail to start on centos7
- ...
- 01:54 PM Bug #12598 (Resolved): LibCephFS.GetPoolId failure
- ...
- 11:51 AM Bug #12578 (Fix Under Review): multimds: mds/SnapRealm.cc: 238: FAILED assert(open)
- https://github.com/ceph/ceph/pull/5466
- 08:52 AM Backport #12590 (Resolved): "ceph mds add_data_pool" check for EC pool is wrong
- https://github.com/ceph/ceph/pull/5766
08/03/2015
- 09:03 PM Bug #12578 (Resolved): multimds: mds/SnapRealm.cc: 238: FAILED assert(open)
- 2015-08-02T07:54:05.552 INFO:tasks.ceph.mds.d.mira041.stderr:mds/SnapRealm.cc: In function 'void SnapRealm::check_cac...
- 07:16 PM Bug #12222 (Resolved): MDSMonitor: set max_mds doesn't respect MAX_MDS
- 12:56 PM Bug #12222 (Fix Under Review): MDSMonitor: set max_mds doesn't respect MAX_MDS
- https://github.com/ceph/ceph/pull/5397
07/31/2015
- 01:05 PM Feature #12552: qa: test cephfs over cache tier in fs suite
- see #12551
- 01:04 PM Feature #12552 (Rejected): qa: test cephfs over cache tier in fs suite
- see
- 11:42 AM Bug #12547 (New): CEPH_QA_SUITE/AARCH64: marginal:mds_restart fails
- The following test fails on AArch64:
marginal:mds_restart/{clusters/one_mds.yaml tasks/restart-workunit-backtraces.y... - 11:37 AM Bug #12546 (New): CEPH_QA_SUITE/AARCH64: marginal:fs-misc fail
- The following test fails on AArch64:
marginal:fs-misc/{clusters/two_clients.yaml fs/btrfs.yaml tasks/locktest.yaml}
... - 07:52 AM Bug #12088 (Resolved): cephfs client crash after enable readahead mechanism through setting conf ...
- 07:52 AM Backport #12319 (Resolved): cephfs client crash after enable readahead mechanism through setting ...
07/30/2015
- 08:49 AM Bug #10944: Deadlock, MDS logs "slow request", getattr pAsLsXsFs failed to rdlock
- will do
- 07:45 AM Bug #10944: Deadlock, MDS logs "slow request", getattr pAsLsXsFs failed to rdlock
- Hi Yan,
Those patches look good after performing test for a long time. Will you commit the patches to both ceph an...
07/29/2015
- 12:32 PM Feature #4161 (Resolved): MDS: add file layout to head object
- ...
- 12:31 PM Bug #11985 (Resolved): MDS asserts in objecter when transitioning from replay to DNE
- ...
07/28/2015
- 03:18 PM Backport #12500 (In Progress): segfault launching ceph-fuse with bad --name
- 10:37 AM Backport #12500 (Resolved): segfault launching ceph-fuse with bad --name
- https://github.com/ceph/ceph/pull/5382
- 03:18 PM Backport #12499 (In Progress): ceph-fuse 0.94.2-1trusty segfaults / aborts
- 10:37 AM Backport #12499 (New): ceph-fuse 0.94.2-1trusty segfaults / aborts
- 10:36 AM Backport #12499 (Resolved): ceph-fuse 0.94.2-1trusty segfaults / aborts
- https://github.com/ceph/ceph/pull/5381
- 03:03 PM Bug #12506 (Resolved): "Fuse mount failed to populate" error
- Run: http://pulpito.ceph.com/teuthology-2015-07-27_16:05:09-upgrade:hammer-hammer-distro-basic-vps/
Jobs: ['988514',... - 01:41 PM Bug #11255 (Fix Under Review): nfs: mount failures on ceph-backed NFS share
- 01:40 PM Bug #12426 (Pending Backport): "ceph mds add_data_pool" check for EC pool is wrong
07/23/2015
- 01:57 PM Feature #12445 (New): improved path enforcement for hard links
- The initial path restriction will not work properly for remote links inside the security domain that link to a primar...
- 01:05 PM Bug #12355 (Resolved): MDS assertion during shutdown (MDLog !capped), in TestStrays.test_migratio...
- 09:50 AM Bug #12355 (Fix Under Review): MDS assertion during shutdown (MDLog !capped), in TestStrays.test_...
- https://github.com/ceph/ceph/pull/5324
- 09:46 AM Bug #12426 (Fix Under Review): "ceph mds add_data_pool" check for EC pool is wrong
- https://github.com/ceph/ceph/pull/5323
- 08:18 AM Bug #12209: CephFS should have a complete timeout mechanism to avoid endless waiting or unpredict...
- i think you can send a Pull request on github
- 02:02 AM Feature #12334: nfs-ganesha: handle client cache pressure in NFS Ganesha FSAL
- I finished the final test using 1 Ceph file system client and the fuse interface. I ran the create 5 million file te...
07/22/2015
- 03:38 PM Bug #12417 (Pending Backport): segfault launching ceph-fuse with bad --name
- 12:59 PM Bug #12355 (In Progress): MDS assertion during shutdown (MDLog !capped), in TestStrays.test_migra...
- Reproduced this locally. Fixing...
- 03:22 AM Bug #12088: cephfs client crash after enable readahead mechanism through setting conf option 'cli...
- Zheng Yan wrote:
> https://github.com/ceph/ceph/pull/5222
Hi Yan,
We found a serious problem with this patch w... - 02:00 AM Bug #12354 (Resolved): ceph-fuse crash in ll_fsync, during TestClientRecovery.test_fsync
- fuse fsyncdir callback was added recently
07/21/2015
- 09:43 PM Bug #12426 (Resolved): "ceph mds add_data_pool" check for EC pool is wrong
We have a generic MDSMonitor::_check_pool that's used in fs new, but add_data_pool is just checking if something is...- 03:14 PM Bug #12417 (Fix Under Review): segfault launching ceph-fuse with bad --name
- https://github.com/ceph/ceph/pull/5305
- 10:09 AM Bug #12417 (Resolved): segfault launching ceph-fuse with bad --name
- This is just in a vstart environment -- running ceph-fuse with no --name arg is fine....
- 09:54 AM Bug #11255: nfs: mount failures on ceph-backed NFS share
- Did you mean to reference https://github.com/ceph/teuthology/pull/554, Zheng?
- 09:35 AM Bug #11255: nfs: mount failures on ceph-backed NFS share
- maybe https://github.com/ceph/teuthology/pull/342 can fix this
- 08:37 AM Bug #11255: nfs: mount failures on ceph-backed NFS share
- http://pulpito.ceph.redhat.com/teuthology-2015-07-15_18:10:02-knfs-hammer-testing-basic-magna/153971/
- 09:51 AM Bug #12357: kclient: some kind of hang
- Sorry I missed that. I believe the missing bzip2 is being handled elsewhere by the teuthology guys.
- 02:37 AM Bug #12357 (Won't Fix): kclient: some kind of hang
- ...
- 09:48 AM Bug #12354 (Pending Backport): ceph-fuse crash in ll_fsync, during TestClientRecovery.test_fsync
- I guess this is a long-standing issue, but given CephFS' support state just backporting to hammer should be fine.
- 02:38 AM Bug #12354 (Fix Under Review): ceph-fuse crash in ll_fsync, during TestClientRecovery.test_fsync
- https://github.com/ceph/ceph/pull/5294
- 02:27 AM Bug #12365: kcephfs: hang on umount
- by commit https://github.com/ceph/ceph-client/commit/d2ca589aa92410b6760047dea877ca6edd0b15c7
07/20/2015
- 07:33 PM Bug #12297 (Pending Backport): ceph-fuse 0.94.2-1trusty segfaults / aborts
- Excellent! This is merged to master now.
- 06:45 AM Bug #12297: ceph-fuse 0.94.2-1trusty segfaults / aborts
- Cluster has been processing data without any problem during the weekend. I consider the bug fixed.
- 07:08 PM Feature #12334: nfs-ganesha: handle client cache pressure in NFS Ganesha FSAL
- I am seeing the "Client <foo> failing to respond to cache pressure" warning using the Ceph Kernel driver after creat...
07/19/2015
- 03:52 AM Feature #12334: nfs-ganesha: handle client cache pressure in NFS Ganesha FSAL
- I ran the same 5 million file create test using a cifs mount instead of a NFS mount and did not see the "Client <foo>...
07/17/2015
- 04:26 PM Bug #12354: ceph-fuse crash in ll_fsync, during TestClientRecovery.test_fsync
- I've got the core file and appropriate packages on vpm119 if you want an environment to look at it with.
- 04:25 PM Bug #12354: ceph-fuse crash in ll_fsync, during TestClientRecovery.test_fsync
- I'm not finding where it's gone wrong, but I think this must be an issue with the new refcounting. Please take a look...
- 03:40 PM Bug #12354: ceph-fuse crash in ll_fsync, during TestClientRecovery.test_fsync
- Or more likely we're freeing the Fh inappropriately and reusing the memory for a dir_result_t, since we control the l...
- 03:36 PM Bug #12354: ceph-fuse crash in ll_fsync, during TestClientRecovery.test_fsync
- Looking at the core dump, it's crashing on dereferencing a NULL Inode* contained in the passed-in Fh structure. But t...
- 01:30 PM Bug #12354 (In Progress): ceph-fuse crash in ll_fsync, during TestClientRecovery.test_fsync
- 01:07 PM Bug #12344: libcephfs-java/test.sh: com.ceph.fs.CephMountTest fails
- We haven't seen this in any of our regular runs, which makes me think it's a config issue in your OpenStack setup. Se...
- 11:42 AM Bug #12297: ceph-fuse 0.94.2-1trusty segfaults / aborts
- Branch is deployed to the compute cluster host and the application causing the problem is started.
Debug output a... - 10:00 AM Bug #12365 (In Progress): kcephfs: hang on umount
07/16/2015
- 03:57 PM Bug #12297: ceph-fuse 0.94.2-1trusty segfaults / aborts
- https://github.com/ceph/ceph/pull/5260
- 02:42 PM Bug #12297: ceph-fuse 0.94.2-1trusty segfaults / aborts
- Burkhard, if you can test out the hammer-12297 branch on one of the boxes seeing this bug and find out if that fixes ...
- 12:07 PM Bug #12297 (Fix Under Review): ceph-fuse 0.94.2-1trusty segfaults / aborts
- I think I have a fix for this issue but I want to get a few eyes on it before handing it off for user testing.
- 03:51 PM Bug #12365 (Resolved): kcephfs: hang on umount
- http://pulpito.ceph.com/teuthology-2015-07-13_23:08:02-kcephfs-next-testing-basic-multi/972963/
There was an error... - 02:37 PM Bug #12357 (Won't Fix): kclient: some kind of hang
- http://pulpito.ceph.redhat.com/teuthology-2015-07-12_18:08:02-kcephfs-hammer-testing-basic-magna/151448/
It looks ... - 02:16 PM Bug #12355: MDS assertion during shutdown (MDLog !capped), in TestStrays.test_migration_on_shutdown
- See also http://pulpito.ceph.com/teuthology-2015-07-03_23:04:02-fs-master---basic-multi/959994/
- 01:56 PM Bug #12355 (Resolved): MDS assertion during shutdown (MDLog !capped), in TestStrays.test_migratio...
/a/teuthology-2015-07-13_23:04:03-fs-master---basic-multi/972783
mds.a crashed...- 01:53 PM Bug #12354 (Resolved): ceph-fuse crash in ll_fsync, during TestClientRecovery.test_fsync
teuthology-2015-07-13_23:04:03-fs-master---basic-multi/972688/
ceph-client.0.4375.log...- 08:59 AM Backport #12350 (Resolved): Provided logrotate setup does not handle ceph-fuse correctly
- https://github.com/ceph/ceph/pull/7110
- 02:22 AM Bug #12088: cephfs client crash after enable readahead mechanism through setting conf option 'cli...
- Hi, Yan. I think it is also unsafe in the _read_async code when this patch applied. When thread1 wait on read, threa...
07/15/2015
- 09:47 PM Bug #12344 (Can't reproduce): libcephfs-java/test.sh: com.ceph.fs.CephMountTest fails
- This is on an OpenStack virtual machine with 8GB RAM, 40GB disk, 1CPU....
- 02:43 PM Bug #12321: MDS crash when try to connect clients
- After you've destroyed the MDS's record of client sessions, it's important to make sure that those clients don't exis...
- 05:18 AM Bug #12321: MDS crash when try to connect clients
- zcc icy wrote:
> John Spray wrote:
> > This indicates an inconsistency between the inodes apparently allocated to t... - 04:38 AM Bug #12321: MDS crash when try to connect clients
- John Spray wrote:
> This indicates an inconsistency between the inodes apparently allocated to the session, and thos... - 04:34 AM Bug #12321: MDS crash when try to connect clients
- Greg Farnum wrote:
> What's the disaster that happened? Did you lose some of your RADOS objects?
I deploy 3 mon &... - 11:21 AM Feature #12334: nfs-ganesha: handle client cache pressure in NFS Ganesha FSAL
- Note that NFS-Ganesha created significantly more inodes than the cache size limit before it had too many pinned. So i...
- 07:31 AM Feature #12334 (Resolved): nfs-ganesha: handle client cache pressure in NFS Ganesha FSAL
- Reported by Eric Eastman on ceph-users: http://lists.ceph.com/pipermail/ceph-users-ceph.com/2015-July/003000.html
...
07/14/2015
- 04:53 PM Bug #12297: ceph-fuse 0.94.2-1trusty segfaults / aborts
- Okay, I dug into this today and I think our readdir is a bit broken (obviously). In particular, we're dropping the cl...
- 09:48 AM Bug #12297: ceph-fuse 0.94.2-1trusty segfaults / aborts
- We've got logs and core file via ceph-post-file
c00fb309-e2a1-482c-b093-1733affb2901 gzipped core dump
60fbd6a7... - 08:03 AM Bug #12297: ceph-fuse 0.94.2-1trusty segfaults / aborts
- I can't figure out what's wrong neither
- 02:43 PM Bug #12321: MDS crash when try to connect clients
- This indicates an inconsistency between the inodes apparently allocated to the session, and those marked as free in t...
- 02:19 PM Bug #12321: MDS crash when try to connect clients
- What's the disaster that happened? Did you lose some of your RADOS objects?
- 08:23 AM Bug #12321: MDS crash when try to connect clients
- I have 3 mon 9 osd .
when I recovery after the disaster happened. I found that mds can't run.
log is in the file. - 08:19 AM Bug #12321 (Can't reproduce): MDS crash when try to connect clients
- I have 3 mon 9 osd .
when I recovery after the disaster happened. I found that mds can't run.
log :
2015-... - 10:45 AM Bug #12088: cephfs client crash after enable readahead mechanism through setting conf option 'cli...
- Backports team: Zheng created a backport PR at https://github.com/ceph/ceph/pull/5222, but let's let it bake a bit in...
- 06:45 AM Bug #12088: cephfs client crash after enable readahead mechanism through setting conf option 'cli...
- https://github.com/ceph/ceph/pull/5222
- 07:39 AM Backport #12319 (In Progress): cephfs client crash after enable readahead mechanism through setti...
- 07:38 AM Backport #12319 (Resolved): cephfs client crash after enable readahead mechanism through setting ...
- https://github.com/ceph/ceph/pull/5222 and https://github.com/ceph/ceph/pull/5427
- 03:52 AM Bug #10944: Deadlock, MDS logs "slow request", getattr pAsLsXsFs failed to rdlock
- Hi Yan,
We have tested both patches for few days and no above issue happened again. We will perform some other tes...
07/13/2015
- 02:40 PM Bug #12297: ceph-fuse 0.94.2-1trusty segfaults / aborts
- Zheng, can you take a look at this and see if you need the log or can identify the issue? I presume something's gone ...
- 02:33 PM Bug #12297: ceph-fuse 0.94.2-1trusty segfaults / aborts
- (gdb) p dir
$2 = (Dir *) 0x7f709000f9a0
(gdb) p dir->dentry_list
$3 = {_front = 0x7f6f8002a260, _back = 0x7f70b83a... - 02:25 PM Bug #12297: ceph-fuse 0.94.2-1trusty segfaults / aborts
- Okay, I think I see what happened, one more please.
[run gdb]
f 4
p dir
p dir->dentry_list
And if you could ... - 02:18 PM Bug #12297: ceph-fuse 0.94.2-1trusty segfaults / aborts
- (gdb) f 4
#4 Client::_readdir_cache_cb (this=this@entry=0x38a8ab0, dirp=dirp@entry=0x7f6f7c018400, cb=cb@entry=
... - 02:14 PM Bug #12297: ceph-fuse 0.94.2-1trusty segfaults / aborts
- That's pretty bizarre. Signal 7 is SIGBUS "Bus error (bad memory access)", which is separate from SIGSEGV. And given ...
- 01:48 PM Bug #12297: ceph-fuse 0.94.2-1trusty segfaults / aborts
- (gdb) bt
#0 0x00007f70d7bf020b in raise (sig=7)
at ../nptl/sysdeps/unix/sysv/linux/pt-raise.c:37
#1 0x000000... - 11:12 AM Bug #12297: ceph-fuse 0.94.2-1trusty segfaults / aborts
- You mean the ulimit for dumping the core file? That will depend on how you're invoking it; if you're just running "ce...
- 11:04 AM Bug #12297: ceph-fuse 0.94.2-1trusty segfaults / aborts
- Sorry, no core file available. I have a number of hosts with CephFS running without any problems; some other hosts sh...
- 10:48 AM Bug #12297: ceph-fuse 0.94.2-1trusty segfaults / aborts
- Do you have a core file from this that you can look at with gdb, and the ceph debug packages installed? (You can pull...
- 10:33 AM Bug #12297 (Resolved): ceph-fuse 0.94.2-1trusty segfaults / aborts
- Running CephFS for data volumes on a compute cluster.
ceph-fuse aborts and leaves the mount point unusable (transp... - 12:40 PM Bug #12088: cephfs client crash after enable readahead mechanism through setting conf option 'cli...
- ...
- 12:40 PM Bug #12088 (Pending Backport): cephfs client crash after enable readahead mechanism through setti...
- I imagine we probably want backports to firefly + hammer?
- 02:24 AM Fix #12296 (Rejected): cephfs-hadoop: do not stash libcephfs.jar in git repo
- The libcephfs.jar artifact should not be stored in the cephfs-hadoop git repository. Ideally it lands in Maven Centra...
Also available in: Atom