Project

General

Profile

Activity

From 08/04/2015 to 09/02/2015

09/02/2015

03:13 PM Bug #12896: EIO in multiple_rsync.sh
Looking at the paths, rsync *seems* to be complaining about the local files (in /tmp). Maybe it's just a bad test node. John Spray
01:21 PM Bug #12896: EIO in multiple_rsync.sh
this failure looks weired. there is no ll_read entry in the client log, there are ll_write enties, but no error. Zheng Yan
10:42 AM Bug #11746 (Resolved): cephfs Dumper tries to load whole journal into memory at once
Loïc Dachary
10:35 AM Backport #12098 (Resolved): kernel_untar_build fails on EL7
Loïc Dachary
10:35 AM Backport #11999 (Resolved): cephfs Dumper tries to load whole journal into memory at once
Loïc Dachary
08:27 AM Backport #12590 (In Progress): "ceph mds add_data_pool" check for EC pool is wrong
Loïc Dachary

09/01/2015

06:03 PM Bug #12909 (Resolved): cmake: client/fuse_ll.cc can't locate fuse_lowlevel.h
"Commit f064e90ae554b64741284ef1cdf8a00bb7b4a312":https://github.com/ceph/ceph/commit/f064e90ae554b64741284ef1cdf8a00... Casey Bodley
04:23 PM Bug #12820: stuck looping on 'ls /sys/fs/fuse/connections'
Greg Farnum
10:33 AM Bug #12820 (Fix Under Review): stuck looping on 'ls /sys/fs/fuse/connections'
https://github.com/ceph/ceph-qa-suite/pull/551 John Spray
12:30 PM Bug #12776 (Fix Under Review): qa: standby MDS not shutting down, "reached maximum tries (50) aft...
https://github.com/ceph/ceph/pull/5739 John Spray
11:22 AM Bug #12896 (Rejected): EIO in multiple_rsync.sh

http://pulpito.ceph.com/teuthology-2015-08-28_23:04:01-fs-master---basic-multi/1037227/...
John Spray
11:12 AM Bug #12895 (Can't reproduce): Failure in TestClusterFull.test_barrier

teuthology-2015-08-24_23:04:02-fs-master---basic-multi/1030586/
John Spray

08/31/2015

02:58 PM Bug #12875 (Can't reproduce): LibCephFS.LibCephFS.InterProcessLocking segment fault.
... Sage Weil
02:29 PM Bug #12806 (Fix Under Review): nfs restart failures
https://github.com/ceph/ceph-qa-suite/pull/550 Zheng Yan
09:16 AM Bug #12806: nfs restart failures
looks like it's added by ceph-qa-suite/tasks/qemu.py. Zheng Yan
12:53 PM Bug #11482: kclient: intermittent log warnings "client.XXXX isn't responding to mclientcaps(revoke)"
For some reason the override we provided isn't being added to the configs, I created #12869 for that. :/ Greg Farnum

08/29/2015

09:20 AM Bug #12344 (Can't reproduce): libcephfs-java/test.sh: com.ceph.fs.CephMountTest fails
Loïc Dachary

08/28/2015

01:13 PM Feature #10369: qa-suite: detect unexpected MDS failovers and daemon crashes
We just keep re-creating this feature: #12821 Greg Farnum
01:13 PM Bug #12821 (Duplicate): mds_thrasher: handle MDSes failing on startup
#10369 Greg Farnum
01:11 PM Bug #12821: mds_thrasher: handle MDSes failing on startup
This is kind of a special case of http://tracker.ceph.com/issues/10369 -- there is a more general need for something ... John Spray
11:31 AM Bug #12821 (Duplicate): mds_thrasher: handle MDSes failing on startup
http://pulpito.ceph.com/teuthology-2015-08-21_23:04:01-fs-master---basic-multi/1026045/... Greg Farnum
01:02 PM Bug #12822 (Resolved): ceph-fuse crash in test_client_recovery
If we actually see that it's crashing with the timeout we can reopen this. Greg Farnum
12:58 PM Bug #12822: ceph-fuse crash in test_client_recovery
I would expect us to see a backtrace from ceph-fuse stderr in the case of an actual crash. Seems more like the clien... John Spray
11:53 AM Bug #12822 (Resolved): ceph-fuse crash in test_client_recovery
http://pulpito.ceph.com/teuthology-2015-08-17_23:04:01-fs-master---basic-multi/1020395/
Sadly there are absolutely...
Greg Farnum
12:12 PM Feature #12823 (Rejected): cephfs_test_runner: print test names when executing them
They're already logged, like this:... John Spray
11:57 AM Feature #12823 (Rejected): cephfs_test_runner: print test names when executing them
It looks like we don't print out test names. When running through a whole suite that makes telling where we are in a ... Greg Farnum
12:10 PM Bug #12776: qa: standby MDS not shutting down, "reached maximum tries (50) after waiting for 300 ...
Actually, I just tried sending SIGTERM to a standby mds here, and it's getting stuck too. John Spray
12:08 PM Bug #12776: qa: standby MDS not shutting down, "reached maximum tries (50) after waiting for 300 ...
It's getting the signal, but not making it through shutdown:... John Spray
11:48 AM Bug #12820: stuck looping on 'ls /sys/fs/fuse/connections'
http://pulpito.ceph.com/teuthology-2015-08-17_23:04:01-fs-master---basic-multi/1020354/ Greg Farnum
11:24 AM Bug #12820 (Resolved): stuck looping on 'ls /sys/fs/fuse/connections'
http://pulpito.ceph.com/teuthology-2015-08-21_23:04:01-fs-master---basic-multi/1025967/... Greg Farnum
11:47 AM Bug #12612 (Can't reproduce): fuse jobs fail to start on centos7
Okay, haven't seen this particular one again, just the new one #12820. Greg Farnum
10:11 AM Bug #12808: smbtorture failure on scan-pipe
It doesn't look like that in the error logs to me; I think it just failed to allocate — but perhaps I'm misreading th... Greg Farnum
02:19 AM Bug #12808: smbtorture failure on scan-pipe
this test case makes smbd allocate tens of GBs memory. maybe smbd got killed during the test Zheng Yan
10:04 AM Bug #12806: nfs restart failures
Zheng, do we have any idea how the machines are getting into that duplicated export state? It looks pretty clear that... Greg Farnum
01:35 AM Bug #12806 (Resolved): nfs restart failures
Zheng Yan
01:35 AM Bug #12806: nfs restart failures
... Zheng Yan
09:38 AM Bug #12657 (Can't reproduce): Failure in TestStrays.test_ops_throttle
Hmm, now this looks like a teuthology burp. The stats polling is meant to happen every second, but in this instance ... John Spray
08:18 AM Bug #12777: qa: leftover files in cephtest directory
merged to next and master. John Spray
01:56 AM Bug #12777: qa: leftover files in cephtest directory
Zheng Yan

08/27/2015

04:55 PM Bug #12777 (Fix Under Review): qa: leftover files in cephtest directory

Oops, this is CephFSTestCase.tearDown not getting called (so there's still a client mount, so the dir can't be remo...
John Spray
01:34 PM Bug #12777: qa: leftover files in cephtest directory
This shows up in the logs as 'rmdir -- /home/ubuntu/cephtest' Greg Farnum
04:20 PM Bug #12806: nfs restart failures
iirc starting the nfs service on el7 depends on rpcbind, nfs-lock already being up, so that might be what's missing here John Spray
01:19 PM Bug #12806 (Resolved): nfs restart failures
... Greg Farnum
01:46 PM Bug #12808 (New): smbtorture failure on scan-pipe
Log summary line: "Command failed on burnupi59 with status 137: 'TESTDIR=/home/ubuntu/cephtest bash -s'"... Greg Farnum
01:35 PM Bug #12807 (Duplicate): rmdir cephtest failing
#12777 Greg Farnum
01:30 PM Bug #12807 (Duplicate): rmdir cephtest failing
http://pulpito.ceph.com/teuthology-2015-08-22_23:04:02-fs-next---basic-multi/1027445/
http://pulpito.ceph.com/teutho...
Greg Farnum

08/25/2015

02:00 PM Bug #11789: knfs mount fails with "getfh failed: Function not implemented"
We think this is the NFS kernel module not being loaded. If this is still happening we need to figure out why. Greg Farnum
01:58 PM Bug #12653 (Fix Under Review): fuse mounted file systems fails SAMBA CTDB ping_pong rw test with ...
Greg Farnum
01:56 PM Bug #11783 (Fix Under Review): protocol: flushing caps on MDS restart can go bad
Greg Farnum
01:52 PM Bug #11784 (Can't reproduce): ceph-fuse hang on unmount (stuck dentry refs)
Sage Weil
01:51 PM Bug #9994: ceph-qa-suite: nfs mount timeouts
Greg Farnum
01:51 PM Bug #12365 (Resolved): kcephfs: hang on umount
Haven't seen this since then. Greg Farnum
12:59 PM Bug #12777: qa: leftover files in cephtest directory
http://pulpito.ceph.com/teuthology-2015-08-21_23:04:01-fs-master---basic-multi/1026047/ Greg Farnum
12:13 PM Bug #12777 (Resolved): qa: leftover files in cephtest directory
http://pulpito.ceph.com/teuthology-2015-08-17_23:04:01-fs-master---basic-multi/1020426/
http://pulpito.ceph.com/teut...
Greg Farnum
12:33 PM Bug #11482: kclient: intermittent log warnings "client.XXXX isn't responding to mclientcaps(revoke)"
http://pulpito.ceph.com/teuthology-2015-08-17_23:08:05-kcephfs-master-testing-basic-multi/1020518/
Doesn't have th...
Greg Farnum
12:10 PM Bug #12776 (Resolved): qa: standby MDS not shutting down, "reached maximum tries (50) after waiti...
http://pulpito.ceph.com/teuthology-2015-08-17_23:04:01-fs-master---basic-multi/1020415/
The standby MDS doesn't lo...
Greg Farnum

08/24/2015

06:45 PM Bug #12715 (Resolved): "[ERR] bad backtrace on dir ino 600" in cluster log"
Greg Farnum
06:42 PM Bug #12715: "[ERR] bad backtrace on dir ino 600" in cluster log"
https://github.com/ceph/ceph-qa-suite/pull/539 Yuri Weinstein
06:36 PM Bug #12715: "[ERR] bad backtrace on dir ino 600" in cluster log"
whitelist "bad backtrace on dir ino" warning message (per irc chat with Greg) Yuri Weinstein
05:30 PM Bug #12715: "[ERR] bad backtrace on dir ino 600" in cluster log"
Also in run:
http://pulpito.ceph.com/teuthology-2015-08-21_08:42:54-upgrade:firefly-x-hammer-distro-basic-vps/
Jobs...
Yuri Weinstein
03:37 AM Bug #12715: "[ERR] bad backtrace on dir ino 600" in cluster log"
the test uses 0.80.4 (7c241cfaa6c8c068bc9da8578ca00b9f4fc7567f). the newest firefly include the fix (commit a5970963) Zheng Yan
01:22 PM Bug #12710 (Resolved): fsstress.sh fails
Zheng Yan
01:22 PM Bug #12709 (Resolved): hammer chmod.sh fails
fixed by commit 81a311a744987564b70852fdacfd915523c73b5d Zheng Yan
01:21 PM Bug #12711 (Resolved): mds get damaged
Zheng Yan
08:36 AM Bug #12676 (Resolved): MDSMap assertion in MDCache::trim (multimds)
... John Spray
08:33 AM Bug #12321 (Can't reproduce): MDS crash when try to connect clients
John Spray
03:52 AM Bug #12321: MDS crash when try to connect clients
John Spray wrote:
> Hi, do you have any updates for us on this? If the system is unavailable for any more debug the...
zcc icy
06:30 AM Bug #12753 (Resolved): cls_cephfs_client encodes time_t directly
fixed by 1213dde3d207d0d91ccecfca4dd6af1bdee0ed65 Zheng Yan

08/21/2015

09:40 PM Bug #12753 (Resolved): cls_cephfs_client encodes time_t directly
fail to build on i386.
we should never encode time_t directly.. cast to uint32_t or uint64_t so it is sized explic...
Sage Weil
08:54 PM Bug #12715: "[ERR] bad backtrace on dir ino 600" in cluster log"
Zheng, Sage's mentioned that this may have been fixed by you, can you take a look? Yuri Weinstein
08:19 PM Bug #12715: "[ERR] bad backtrace on dir ino 600" in cluster log"
This is an old bug, right? We should just whitelist this? Sage Weil
07:06 PM Bug #12715: "[ERR] bad backtrace on dir ino 600" in cluster log"
Run http://pulpito.ceph.com/teuthology-2015-08-21_08:42:54-upgrade:firefly-x-hammer-distro-basic-vps/
Jobs: 1024928,...
Yuri Weinstein
05:44 AM Bug #12732: very slow read when a file has holes.
'(rc < 0 && rc != -ENOENT)' should work. please send a patch to ceph-devel@vger.kernel.org Zheng Yan

08/20/2015

10:07 AM Bug #12732 (Resolved): very slow read when a file has holes.
When a file in cephfs has holes, reading this file is very slow.
This problem can be reproduced by the commands bel...
caifeng zhu

08/19/2015

03:27 PM Bug #12727 (Duplicate): fsstress.sh failed in powercycle suite
dup of #12710 Zheng Yan
03:03 PM Bug #12727 (Duplicate): fsstress.sh failed in powercycle suite
Run: http://pulpito.ceph.com/teuthology-2015-08-18_09:06:57-powercycle-hammer-testing-basic-multi/
Job: 1020696
Log...
Yuri Weinstein

08/18/2015

12:07 PM Bug #12710: fsstress.sh fails
commit:47519365484056e1731cac54cce835332d258121 Greg Farnum
07:28 AM Bug #12710 (Fix Under Review): fsstress.sh fails
https://github.com/ceph/ceph/pull/5595 Zheng Yan
12:01 PM Bug #12711: mds get damaged
merged in commit:3cfb7e4ccc08a67ceec73ee684049320c75e9bb2 Greg Farnum
06:59 AM Bug #12711 (Fix Under Review): mds get damaged
https://github.com/ceph/ceph/pull/5594 Zheng Yan

08/17/2015

08:14 PM Bug #12715 (Resolved): "[ERR] bad backtrace on dir ino 600" in cluster log"
Run: http://pulpito.ceph.com/teuthology-2015-08-14_16:56:20-upgrade:firefly-x-hammer-distro-basic-multi/
Job: 101471...
Yuri Weinstein
01:06 PM Bug #12711 (Resolved): mds get damaged
http://pulpito.ceph.com/teuthology-2015-08-10_23:08:02-kcephfs-master-testing-basic-multi/1010323/
It's easy to re...
Zheng Yan
08:06 AM Bug #12710 (Resolved): fsstress.sh fails
see quite a lot fsstress failures. one of them is http://qa-proxy.ceph.com/teuthology/teuthology-2015-08-10_23:04:02-... Zheng Yan
07:50 AM Bug #11783: protocol: flushing caps on MDS restart can go bad
see this again http://pulpito.ceph.com/teuthology-2015-08-11_23:04:02-fs-next---basic-multi/1011375 Zheng Yan
03:49 AM Bug #12709 (Resolved): hammer chmod.sh fails
http://magna002.ceph.redhat.com/teuthology-2015-08-13_18:04:02-fs-hammer---basic-magna/173814/teuthology.log... Zheng Yan

08/14/2015

08:48 AM Bug #12676 (Fix Under Review): MDSMap assertion in MDCache::trim (multimds)
Zheng Yan
08:47 AM Bug #12676: MDSMap assertion in MDCache::trim (multimds)
https://github.com/ceph/ceph/pull/5583 Zheng Yan

08/13/2015

02:48 PM Bug #12598 (Resolved): LibCephFS.GetPoolId failure
commit:4d4fe9dbc0eb0d0eaa9a608474fecc892626f542 Sage Weil

08/12/2015

01:22 PM Bug #12674: Semi-reproducible crash of ceph-fuse
Some additional info which might be useful:
* We are no longer 100% certain that the crashes relate to renames. The ...
Jörg Henne
01:19 PM Bug #12674: Semi-reproducible crash of ceph-fuse
Zheng Yan wrote:
> the log seems like memory allocation failure. how many memory does your machine have?
The VM h...
Jörg Henne
12:45 PM Bug #12674: Semi-reproducible crash of ceph-fuse
the log seems like memory allocation failure. how many memory does your machine have? Zheng Yan
10:22 AM Bug #12674 (Resolved): Semi-reproducible crash of ceph-fuse
we are running ceph version 0.94.2 with a cephfs mounted using ceph-fuse on
Ubuntu 14.04 LTS. I think we have found ...
Jörg Henne
01:18 PM Bug #12676 (Resolved): MDSMap assertion in MDCache::trim (multimds)
... John Spray
12:27 PM Bug #12612: fuse jobs fail to start on centos7
Works for me. Let's see if we see more instances I guess. John Spray
11:12 AM Bug #12612: fuse jobs fail to start on centos7
Hmm. Just ran a job using fuse and it passed. Trying again with this exact config. John Spray
06:23 AM Bug #12617 (Resolved): Missing handling of supplementary groups in ceph-fuse -> rendering it usel...
Zheng Yan
02:42 AM Bug #12617: Missing handling of supplementary groups in ceph-fuse -> rendering it useless for sha...
ignore client check patch makes ceph-fuse work, the get_grouplist patch makes programs that directly use libcephfs wo... Zheng Yan

08/11/2015

09:51 PM Feature #12671: Enforce cache limit during dirfrag load during open_ino (during rejoin)
The source of this observation was https://www.mail-archive.com/ceph-users@lists.ceph.com/msg22235.html
In this in...
John Spray
09:49 PM Feature #12671 (New): Enforce cache limit during dirfrag load during open_ino (during rejoin)

When clients replay requests referring to inodes not found in cache, the inode numbers are stashed for loading late...
John Spray
02:01 PM Bug #12617: Missing handling of supplementary groups in ceph-fuse -> rendering it useless for sha...
Do we need the Client.cc patch here, or just the pull request (https://github.com/ceph/ceph/pull/5480), or both? John Spray
02:00 PM Bug #12321: MDS crash when try to connect clients
Hi, do you have any updates for us on this? If the system is unavailable for any more debug then we'll close this. John Spray
01:55 PM Bug #12619 (Resolved): valgrind errors in fs suite
Sage Weil
01:32 PM Bug #11482: kclient: intermittent log warnings "client.XXXX isn't responding to mclientcaps(revoke)"
We need to turn on MDS logs to get a more useful bit of information John Spray
09:07 AM Bug #12209: CephFS should have a complete timeout mechanism to avoid endless waiting or unpredict...
This issue is covered in http://tracker.ceph.com/issues/10944, and we have tested the two patches, which works well. ... Wenjun Huang

08/10/2015

09:07 PM Bug #11482: kclient: intermittent log warnings "client.XXXX isn't responding to mclientcaps(revoke)"
Reopening this because we're still seeing it on kcephfs, unknown whether underlying cause is in the same area:
http:...
John Spray
06:40 PM Bug #11255: nfs: mount failures on ceph-backed NFS share
http://pulpito.ceph.com/teuthology-2015-08-07_23:10:02-knfs-master-testing-basic-multi/1006003/ John Spray
06:31 PM Bug #12657: Failure in TestStrays.test_ops_throttle
This purged 1600 strays in 6 seconds (and therefore wasn't busy doing so for long enough to hit the required high wat... John Spray
06:30 PM Bug #12657 (Can't reproduce): Failure in TestStrays.test_ops_throttle

http://pulpito.ceph.com/teuthology-2015-08-07_23:04:03-fs-master---basic-multi/1005757/
John Spray
08:03 AM Bug #12653: fuse mounted file systems fails SAMBA CTDB ping_pong rw test with v9.0.2
This is a ceph-fuse bug, running the test directly on ceph-fuse mount can reproduce the bug. The bug is because that ... Zheng Yan
03:37 AM Bug #12653 (Resolved): fuse mounted file systems fails SAMBA CTDB ping_pong rw test with v9.0.2
While working with the SAMBA CTDB ping_pong test and the Ceph file system, I noticed that file systems mounted using ... Eric Eastman

08/07/2015

08:42 AM Bug #10944: Deadlock, MDS logs "slow request", getattr pAsLsXsFs failed to rdlock
please provide backtrace of the crash Zheng Yan
07:33 AM Bug #10944: Deadlock, MDS logs "slow request", getattr pAsLsXsFs failed to rdlock
Zheng Yan wrote:
> will do
HI,i use your patch in ceph dokan client. and seems cause to client assert crash.
D...
pengcheng lin
02:26 AM Bug #12189 (Won't Fix): Editing / Creating files fails for NFS-over-CephFS on EC pool with cache ...
Zheng Yan

08/06/2015

04:25 PM Bug #12189: Editing / Creating files fails for NFS-over-CephFS on EC pool with cache tier
I was able to update the kernel on the NFS server to version 4.1.4 today, which also allows me to use the kernel clie... Burkhard Linke
11:16 AM Bug #12619: valgrind errors in fs suite
Zheng Yan
08:15 AM Bug #12619 (Fix Under Review): valgrind errors in fs suite
https://github.com/ceph/ceph/pull/5487 Zheng Yan
09:14 AM Bug #12617: Missing handling of supplementary groups in ceph-fuse -> rendering it useless for sha...
Burkhard Linke wrote:
> The following patch adds correct supplementary group retrieval:
>
> If no callback is def...
Zheng Yan

08/05/2015

02:29 PM Bug #12617: Missing handling of supplementary groups in ceph-fuse -> rendering it useless for sha...
see #3301, and commit:328bf482349485b90911fe8953d8ce5f0769c69e for when this was introduced... hopefully we aren't re... Sage Weil
02:27 PM Bug #12617: Missing handling of supplementary groups in ceph-fuse -> rendering it useless for sha...
The following patch adds correct supplementary group retrieval:
If no callback is defined, the getgrouplist() func...
Burkhard Linke
02:25 PM Bug #12617 (Fix Under Review): Missing handling of supplementary groups in ceph-fuse -> rendering...
https://github.com/ceph/ceph/pull/5480 Zheng Yan
11:30 AM Bug #12617: Missing handling of supplementary groups in ceph-fuse -> rendering it useless for sha...
I checked the kernel code, It seem that kernel also does the permission check. can we just remove the permission in c... Zheng Yan
08:06 AM Bug #12617 (Resolved): Missing handling of supplementary groups in ceph-fuse -> rendering it usel...
During investigating a user's bug report I've stumbled accross
https://github.com/ceph/ceph/blob/master/src/client...
Burkhard Linke
12:35 PM Bug #12619 (Resolved): valgrind errors in fs suite
Unclear if these are really CephFS bugs (the SyscallParam thing especially), but it's our tests they're coming up in.... John Spray
11:32 AM Bug #12578 (Resolved): multimds: mds/SnapRealm.cc: 238: FAILED assert(open)
Zheng Yan
10:54 AM Bug #12612: fuse jobs fail to start on centos7
Have these centos7 nodes run other tests successfully? There's a bunch of spurious failures from missing centos7 pac... John Spray
10:14 AM Bug #12612 (Need More Info): fuse jobs fail to start on centos7
ceph-fuse fails with -ETIMEOUT, no information why timeout happened. In this test, fuse_mount.py kept checking /sys/f... Zheng Yan
07:50 AM Bug #12598 (Fix Under Review): LibCephFS.GetPoolId failure
https://github.com/ceph/ceph/pull/5477 Zheng Yan

08/04/2015

09:49 PM Bug #12612 (Can't reproduce): fuse jobs fail to start on centos7
... Sage Weil
01:54 PM Bug #12598 (Resolved): LibCephFS.GetPoolId failure
... Sage Weil
11:51 AM Bug #12578 (Fix Under Review): multimds: mds/SnapRealm.cc: 238: FAILED assert(open)
https://github.com/ceph/ceph/pull/5466 Zheng Yan
08:52 AM Backport #12590 (Resolved): "ceph mds add_data_pool" check for EC pool is wrong
https://github.com/ceph/ceph/pull/5766 Nathan Cutler
 

Also available in: Atom