Activity
From 08/04/2015 to 09/02/2015
09/02/2015
- 03:13 PM Bug #12896: EIO in multiple_rsync.sh
- Looking at the paths, rsync *seems* to be complaining about the local files (in /tmp). Maybe it's just a bad test node.
- 01:21 PM Bug #12896: EIO in multiple_rsync.sh
- this failure looks weired. there is no ll_read entry in the client log, there are ll_write enties, but no error.
- 10:42 AM Bug #11746 (Resolved): cephfs Dumper tries to load whole journal into memory at once
- 10:35 AM Backport #12098 (Resolved): kernel_untar_build fails on EL7
- 10:35 AM Backport #11999 (Resolved): cephfs Dumper tries to load whole journal into memory at once
- 08:27 AM Backport #12590 (In Progress): "ceph mds add_data_pool" check for EC pool is wrong
09/01/2015
- 06:03 PM Bug #12909 (Resolved): cmake: client/fuse_ll.cc can't locate fuse_lowlevel.h
- "Commit f064e90ae554b64741284ef1cdf8a00bb7b4a312":https://github.com/ceph/ceph/commit/f064e90ae554b64741284ef1cdf8a00...
- 04:23 PM Bug #12820: stuck looping on 'ls /sys/fs/fuse/connections'
- 10:33 AM Bug #12820 (Fix Under Review): stuck looping on 'ls /sys/fs/fuse/connections'
- https://github.com/ceph/ceph-qa-suite/pull/551
- 12:30 PM Bug #12776 (Fix Under Review): qa: standby MDS not shutting down, "reached maximum tries (50) aft...
- https://github.com/ceph/ceph/pull/5739
- 11:22 AM Bug #12896 (Rejected): EIO in multiple_rsync.sh
http://pulpito.ceph.com/teuthology-2015-08-28_23:04:01-fs-master---basic-multi/1037227/...- 11:12 AM Bug #12895 (Can't reproduce): Failure in TestClusterFull.test_barrier
teuthology-2015-08-24_23:04:02-fs-master---basic-multi/1030586/
08/31/2015
- 02:58 PM Bug #12875 (Can't reproduce): LibCephFS.LibCephFS.InterProcessLocking segment fault.
- ...
- 02:29 PM Bug #12806 (Fix Under Review): nfs restart failures
- https://github.com/ceph/ceph-qa-suite/pull/550
- 09:16 AM Bug #12806: nfs restart failures
- looks like it's added by ceph-qa-suite/tasks/qemu.py.
- 12:53 PM Bug #11482: kclient: intermittent log warnings "client.XXXX isn't responding to mclientcaps(revoke)"
- For some reason the override we provided isn't being added to the configs, I created #12869 for that. :/
08/29/2015
08/28/2015
- 01:13 PM Feature #10369: qa-suite: detect unexpected MDS failovers and daemon crashes
- We just keep re-creating this feature: #12821
- 01:13 PM Bug #12821 (Duplicate): mds_thrasher: handle MDSes failing on startup
- #10369
- 01:11 PM Bug #12821: mds_thrasher: handle MDSes failing on startup
- This is kind of a special case of http://tracker.ceph.com/issues/10369 -- there is a more general need for something ...
- 11:31 AM Bug #12821 (Duplicate): mds_thrasher: handle MDSes failing on startup
- http://pulpito.ceph.com/teuthology-2015-08-21_23:04:01-fs-master---basic-multi/1026045/...
- 01:02 PM Bug #12822 (Resolved): ceph-fuse crash in test_client_recovery
- If we actually see that it's crashing with the timeout we can reopen this.
- 12:58 PM Bug #12822: ceph-fuse crash in test_client_recovery
- I would expect us to see a backtrace from ceph-fuse stderr in the case of an actual crash. Seems more like the clien...
- 11:53 AM Bug #12822 (Resolved): ceph-fuse crash in test_client_recovery
- http://pulpito.ceph.com/teuthology-2015-08-17_23:04:01-fs-master---basic-multi/1020395/
Sadly there are absolutely... - 12:12 PM Feature #12823 (Rejected): cephfs_test_runner: print test names when executing them
- They're already logged, like this:...
- 11:57 AM Feature #12823 (Rejected): cephfs_test_runner: print test names when executing them
- It looks like we don't print out test names. When running through a whole suite that makes telling where we are in a ...
- 12:10 PM Bug #12776: qa: standby MDS not shutting down, "reached maximum tries (50) after waiting for 300 ...
- Actually, I just tried sending SIGTERM to a standby mds here, and it's getting stuck too.
- 12:08 PM Bug #12776: qa: standby MDS not shutting down, "reached maximum tries (50) after waiting for 300 ...
- It's getting the signal, but not making it through shutdown:...
- 11:48 AM Bug #12820: stuck looping on 'ls /sys/fs/fuse/connections'
- http://pulpito.ceph.com/teuthology-2015-08-17_23:04:01-fs-master---basic-multi/1020354/
- 11:24 AM Bug #12820 (Resolved): stuck looping on 'ls /sys/fs/fuse/connections'
- http://pulpito.ceph.com/teuthology-2015-08-21_23:04:01-fs-master---basic-multi/1025967/...
- 11:47 AM Bug #12612 (Can't reproduce): fuse jobs fail to start on centos7
- Okay, haven't seen this particular one again, just the new one #12820.
- 10:11 AM Bug #12808: smbtorture failure on scan-pipe
- It doesn't look like that in the error logs to me; I think it just failed to allocate — but perhaps I'm misreading th...
- 02:19 AM Bug #12808: smbtorture failure on scan-pipe
- this test case makes smbd allocate tens of GBs memory. maybe smbd got killed during the test
- 10:04 AM Bug #12806: nfs restart failures
- Zheng, do we have any idea how the machines are getting into that duplicated export state? It looks pretty clear that...
- 01:35 AM Bug #12806 (Resolved): nfs restart failures
- 01:35 AM Bug #12806: nfs restart failures
- ...
- 09:38 AM Bug #12657 (Can't reproduce): Failure in TestStrays.test_ops_throttle
- Hmm, now this looks like a teuthology burp. The stats polling is meant to happen every second, but in this instance ...
- 08:18 AM Bug #12777: qa: leftover files in cephtest directory
- merged to next and master.
- 01:56 AM Bug #12777: qa: leftover files in cephtest directory
08/27/2015
- 04:55 PM Bug #12777 (Fix Under Review): qa: leftover files in cephtest directory
Oops, this is CephFSTestCase.tearDown not getting called (so there's still a client mount, so the dir can't be remo...- 01:34 PM Bug #12777: qa: leftover files in cephtest directory
- This shows up in the logs as 'rmdir -- /home/ubuntu/cephtest'
- 04:20 PM Bug #12806: nfs restart failures
- iirc starting the nfs service on el7 depends on rpcbind, nfs-lock already being up, so that might be what's missing here
- 01:19 PM Bug #12806 (Resolved): nfs restart failures
- ...
- 01:46 PM Bug #12808 (New): smbtorture failure on scan-pipe
- Log summary line: "Command failed on burnupi59 with status 137: 'TESTDIR=/home/ubuntu/cephtest bash -s'"...
- 01:35 PM Bug #12807 (Duplicate): rmdir cephtest failing
- #12777
- 01:30 PM Bug #12807 (Duplicate): rmdir cephtest failing
- http://pulpito.ceph.com/teuthology-2015-08-22_23:04:02-fs-next---basic-multi/1027445/
http://pulpito.ceph.com/teutho...
08/25/2015
- 02:00 PM Bug #11789: knfs mount fails with "getfh failed: Function not implemented"
- We think this is the NFS kernel module not being loaded. If this is still happening we need to figure out why.
- 01:58 PM Bug #12653 (Fix Under Review): fuse mounted file systems fails SAMBA CTDB ping_pong rw test with ...
- 01:56 PM Bug #11783 (Fix Under Review): protocol: flushing caps on MDS restart can go bad
- 01:52 PM Bug #11784 (Can't reproduce): ceph-fuse hang on unmount (stuck dentry refs)
- 01:51 PM Bug #9994: ceph-qa-suite: nfs mount timeouts
- 01:51 PM Bug #12365 (Resolved): kcephfs: hang on umount
- Haven't seen this since then.
- 12:59 PM Bug #12777: qa: leftover files in cephtest directory
- http://pulpito.ceph.com/teuthology-2015-08-21_23:04:01-fs-master---basic-multi/1026047/
- 12:13 PM Bug #12777 (Resolved): qa: leftover files in cephtest directory
- http://pulpito.ceph.com/teuthology-2015-08-17_23:04:01-fs-master---basic-multi/1020426/
http://pulpito.ceph.com/teut... - 12:33 PM Bug #11482: kclient: intermittent log warnings "client.XXXX isn't responding to mclientcaps(revoke)"
- http://pulpito.ceph.com/teuthology-2015-08-17_23:08:05-kcephfs-master-testing-basic-multi/1020518/
Doesn't have th... - 12:10 PM Bug #12776 (Resolved): qa: standby MDS not shutting down, "reached maximum tries (50) after waiti...
- http://pulpito.ceph.com/teuthology-2015-08-17_23:04:01-fs-master---basic-multi/1020415/
The standby MDS doesn't lo...
08/24/2015
- 06:45 PM Bug #12715 (Resolved): "[ERR] bad backtrace on dir ino 600" in cluster log"
- 06:42 PM Bug #12715: "[ERR] bad backtrace on dir ino 600" in cluster log"
- https://github.com/ceph/ceph-qa-suite/pull/539
- 06:36 PM Bug #12715: "[ERR] bad backtrace on dir ino 600" in cluster log"
- whitelist "bad backtrace on dir ino" warning message (per irc chat with Greg)
- 05:30 PM Bug #12715: "[ERR] bad backtrace on dir ino 600" in cluster log"
- Also in run:
http://pulpito.ceph.com/teuthology-2015-08-21_08:42:54-upgrade:firefly-x-hammer-distro-basic-vps/
Jobs... - 03:37 AM Bug #12715: "[ERR] bad backtrace on dir ino 600" in cluster log"
- the test uses 0.80.4 (7c241cfaa6c8c068bc9da8578ca00b9f4fc7567f). the newest firefly include the fix (commit a5970963)
- 01:22 PM Bug #12710 (Resolved): fsstress.sh fails
- 01:22 PM Bug #12709 (Resolved): hammer chmod.sh fails
- fixed by commit 81a311a744987564b70852fdacfd915523c73b5d
- 01:21 PM Bug #12711 (Resolved): mds get damaged
- 08:36 AM Bug #12676 (Resolved): MDSMap assertion in MDCache::trim (multimds)
- ...
- 08:33 AM Bug #12321 (Can't reproduce): MDS crash when try to connect clients
- 03:52 AM Bug #12321: MDS crash when try to connect clients
- John Spray wrote:
> Hi, do you have any updates for us on this? If the system is unavailable for any more debug the... - 06:30 AM Bug #12753 (Resolved): cls_cephfs_client encodes time_t directly
- fixed by 1213dde3d207d0d91ccecfca4dd6af1bdee0ed65
08/21/2015
- 09:40 PM Bug #12753 (Resolved): cls_cephfs_client encodes time_t directly
- fail to build on i386.
we should never encode time_t directly.. cast to uint32_t or uint64_t so it is sized explic... - 08:54 PM Bug #12715: "[ERR] bad backtrace on dir ino 600" in cluster log"
- Zheng, Sage's mentioned that this may have been fixed by you, can you take a look?
- 08:19 PM Bug #12715: "[ERR] bad backtrace on dir ino 600" in cluster log"
- This is an old bug, right? We should just whitelist this?
- 07:06 PM Bug #12715: "[ERR] bad backtrace on dir ino 600" in cluster log"
- Run http://pulpito.ceph.com/teuthology-2015-08-21_08:42:54-upgrade:firefly-x-hammer-distro-basic-vps/
Jobs: 1024928,... - 05:44 AM Bug #12732: very slow read when a file has holes.
- '(rc < 0 && rc != -ENOENT)' should work. please send a patch to ceph-devel@vger.kernel.org
08/20/2015
- 10:07 AM Bug #12732 (Resolved): very slow read when a file has holes.
- When a file in cephfs has holes, reading this file is very slow.
This problem can be reproduced by the commands bel...
08/19/2015
- 03:27 PM Bug #12727 (Duplicate): fsstress.sh failed in powercycle suite
- dup of #12710
- 03:03 PM Bug #12727 (Duplicate): fsstress.sh failed in powercycle suite
- Run: http://pulpito.ceph.com/teuthology-2015-08-18_09:06:57-powercycle-hammer-testing-basic-multi/
Job: 1020696
Log...
08/18/2015
- 12:07 PM Bug #12710: fsstress.sh fails
- commit:47519365484056e1731cac54cce835332d258121
- 07:28 AM Bug #12710 (Fix Under Review): fsstress.sh fails
- https://github.com/ceph/ceph/pull/5595
- 12:01 PM Bug #12711: mds get damaged
- merged in commit:3cfb7e4ccc08a67ceec73ee684049320c75e9bb2
- 06:59 AM Bug #12711 (Fix Under Review): mds get damaged
- https://github.com/ceph/ceph/pull/5594
08/17/2015
- 08:14 PM Bug #12715 (Resolved): "[ERR] bad backtrace on dir ino 600" in cluster log"
- Run: http://pulpito.ceph.com/teuthology-2015-08-14_16:56:20-upgrade:firefly-x-hammer-distro-basic-multi/
Job: 101471... - 01:06 PM Bug #12711 (Resolved): mds get damaged
- http://pulpito.ceph.com/teuthology-2015-08-10_23:08:02-kcephfs-master-testing-basic-multi/1010323/
It's easy to re... - 08:06 AM Bug #12710 (Resolved): fsstress.sh fails
- see quite a lot fsstress failures. one of them is http://qa-proxy.ceph.com/teuthology/teuthology-2015-08-10_23:04:02-...
- 07:50 AM Bug #11783: protocol: flushing caps on MDS restart can go bad
- see this again http://pulpito.ceph.com/teuthology-2015-08-11_23:04:02-fs-next---basic-multi/1011375
- 03:49 AM Bug #12709 (Resolved): hammer chmod.sh fails
- http://magna002.ceph.redhat.com/teuthology-2015-08-13_18:04:02-fs-hammer---basic-magna/173814/teuthology.log...
08/14/2015
- 08:48 AM Bug #12676 (Fix Under Review): MDSMap assertion in MDCache::trim (multimds)
- 08:47 AM Bug #12676: MDSMap assertion in MDCache::trim (multimds)
- https://github.com/ceph/ceph/pull/5583
08/13/2015
- 02:48 PM Bug #12598 (Resolved): LibCephFS.GetPoolId failure
- commit:4d4fe9dbc0eb0d0eaa9a608474fecc892626f542
08/12/2015
- 01:22 PM Bug #12674: Semi-reproducible crash of ceph-fuse
- Some additional info which might be useful:
* We are no longer 100% certain that the crashes relate to renames. The ... - 01:19 PM Bug #12674: Semi-reproducible crash of ceph-fuse
- Zheng Yan wrote:
> the log seems like memory allocation failure. how many memory does your machine have?
The VM h... - 12:45 PM Bug #12674: Semi-reproducible crash of ceph-fuse
- the log seems like memory allocation failure. how many memory does your machine have?
- 10:22 AM Bug #12674 (Resolved): Semi-reproducible crash of ceph-fuse
- we are running ceph version 0.94.2 with a cephfs mounted using ceph-fuse on
Ubuntu 14.04 LTS. I think we have found ... - 01:18 PM Bug #12676 (Resolved): MDSMap assertion in MDCache::trim (multimds)
- ...
- 12:27 PM Bug #12612: fuse jobs fail to start on centos7
- Works for me. Let's see if we see more instances I guess.
- 11:12 AM Bug #12612: fuse jobs fail to start on centos7
- Hmm. Just ran a job using fuse and it passed. Trying again with this exact config.
- 06:23 AM Bug #12617 (Resolved): Missing handling of supplementary groups in ceph-fuse -> rendering it usel...
- 02:42 AM Bug #12617: Missing handling of supplementary groups in ceph-fuse -> rendering it useless for sha...
- ignore client check patch makes ceph-fuse work, the get_grouplist patch makes programs that directly use libcephfs wo...
08/11/2015
- 09:51 PM Feature #12671: Enforce cache limit during dirfrag load during open_ino (during rejoin)
- The source of this observation was https://www.mail-archive.com/ceph-users@lists.ceph.com/msg22235.html
In this in... - 09:49 PM Feature #12671 (New): Enforce cache limit during dirfrag load during open_ino (during rejoin)
When clients replay requests referring to inodes not found in cache, the inode numbers are stashed for loading late...- 02:01 PM Bug #12617: Missing handling of supplementary groups in ceph-fuse -> rendering it useless for sha...
- Do we need the Client.cc patch here, or just the pull request (https://github.com/ceph/ceph/pull/5480), or both?
- 02:00 PM Bug #12321: MDS crash when try to connect clients
- Hi, do you have any updates for us on this? If the system is unavailable for any more debug then we'll close this.
- 01:55 PM Bug #12619 (Resolved): valgrind errors in fs suite
- 01:32 PM Bug #11482: kclient: intermittent log warnings "client.XXXX isn't responding to mclientcaps(revoke)"
- We need to turn on MDS logs to get a more useful bit of information
- 09:07 AM Bug #12209: CephFS should have a complete timeout mechanism to avoid endless waiting or unpredict...
- This issue is covered in http://tracker.ceph.com/issues/10944, and we have tested the two patches, which works well. ...
08/10/2015
- 09:07 PM Bug #11482: kclient: intermittent log warnings "client.XXXX isn't responding to mclientcaps(revoke)"
- Reopening this because we're still seeing it on kcephfs, unknown whether underlying cause is in the same area:
http:... - 06:40 PM Bug #11255: nfs: mount failures on ceph-backed NFS share
- http://pulpito.ceph.com/teuthology-2015-08-07_23:10:02-knfs-master-testing-basic-multi/1006003/
- 06:31 PM Bug #12657: Failure in TestStrays.test_ops_throttle
- This purged 1600 strays in 6 seconds (and therefore wasn't busy doing so for long enough to hit the required high wat...
- 06:30 PM Bug #12657 (Can't reproduce): Failure in TestStrays.test_ops_throttle
http://pulpito.ceph.com/teuthology-2015-08-07_23:04:03-fs-master---basic-multi/1005757/- 08:03 AM Bug #12653: fuse mounted file systems fails SAMBA CTDB ping_pong rw test with v9.0.2
- This is a ceph-fuse bug, running the test directly on ceph-fuse mount can reproduce the bug. The bug is because that ...
- 03:37 AM Bug #12653 (Resolved): fuse mounted file systems fails SAMBA CTDB ping_pong rw test with v9.0.2
- While working with the SAMBA CTDB ping_pong test and the Ceph file system, I noticed that file systems mounted using ...
08/07/2015
- 08:42 AM Bug #10944: Deadlock, MDS logs "slow request", getattr pAsLsXsFs failed to rdlock
- please provide backtrace of the crash
- 07:33 AM Bug #10944: Deadlock, MDS logs "slow request", getattr pAsLsXsFs failed to rdlock
- Zheng Yan wrote:
> will do
HI,i use your patch in ceph dokan client. and seems cause to client assert crash.
D... - 02:26 AM Bug #12189 (Won't Fix): Editing / Creating files fails for NFS-over-CephFS on EC pool with cache ...
08/06/2015
- 04:25 PM Bug #12189: Editing / Creating files fails for NFS-over-CephFS on EC pool with cache tier
- I was able to update the kernel on the NFS server to version 4.1.4 today, which also allows me to use the kernel clie...
- 11:16 AM Bug #12619: valgrind errors in fs suite
- 08:15 AM Bug #12619 (Fix Under Review): valgrind errors in fs suite
- https://github.com/ceph/ceph/pull/5487
- 09:14 AM Bug #12617: Missing handling of supplementary groups in ceph-fuse -> rendering it useless for sha...
- Burkhard Linke wrote:
> The following patch adds correct supplementary group retrieval:
>
> If no callback is def...
08/05/2015
- 02:29 PM Bug #12617: Missing handling of supplementary groups in ceph-fuse -> rendering it useless for sha...
- see #3301, and commit:328bf482349485b90911fe8953d8ce5f0769c69e for when this was introduced... hopefully we aren't re...
- 02:27 PM Bug #12617: Missing handling of supplementary groups in ceph-fuse -> rendering it useless for sha...
- The following patch adds correct supplementary group retrieval:
If no callback is defined, the getgrouplist() func... - 02:25 PM Bug #12617 (Fix Under Review): Missing handling of supplementary groups in ceph-fuse -> rendering...
- https://github.com/ceph/ceph/pull/5480
- 11:30 AM Bug #12617: Missing handling of supplementary groups in ceph-fuse -> rendering it useless for sha...
- I checked the kernel code, It seem that kernel also does the permission check. can we just remove the permission in c...
- 08:06 AM Bug #12617 (Resolved): Missing handling of supplementary groups in ceph-fuse -> rendering it usel...
- During investigating a user's bug report I've stumbled accross
https://github.com/ceph/ceph/blob/master/src/client... - 12:35 PM Bug #12619 (Resolved): valgrind errors in fs suite
- Unclear if these are really CephFS bugs (the SyscallParam thing especially), but it's our tests they're coming up in....
- 11:32 AM Bug #12578 (Resolved): multimds: mds/SnapRealm.cc: 238: FAILED assert(open)
- 10:54 AM Bug #12612: fuse jobs fail to start on centos7
- Have these centos7 nodes run other tests successfully? There's a bunch of spurious failures from missing centos7 pac...
- 10:14 AM Bug #12612 (Need More Info): fuse jobs fail to start on centos7
- ceph-fuse fails with -ETIMEOUT, no information why timeout happened. In this test, fuse_mount.py kept checking /sys/f...
- 07:50 AM Bug #12598 (Fix Under Review): LibCephFS.GetPoolId failure
- https://github.com/ceph/ceph/pull/5477
08/04/2015
- 09:49 PM Bug #12612 (Can't reproduce): fuse jobs fail to start on centos7
- ...
- 01:54 PM Bug #12598 (Resolved): LibCephFS.GetPoolId failure
- ...
- 11:51 AM Bug #12578 (Fix Under Review): multimds: mds/SnapRealm.cc: 238: FAILED assert(open)
- https://github.com/ceph/ceph/pull/5466
- 08:52 AM Backport #12590 (Resolved): "ceph mds add_data_pool" check for EC pool is wrong
- https://github.com/ceph/ceph/pull/5766
Also available in: Atom