Project

General

Profile

Activity

From 05/21/2015 to 06/19/2015

06/19/2015

07:13 PM Backport #12098 (Resolved): kernel_untar_build fails on EL7
https://github.com/ceph/ceph/pull/5119 Loïc Dachary
07:12 PM Backport #12097 (Resolved): kernel_untar_build fails on EL7
https://github.com/ceph/ceph/pull/6000 Loïc Dachary
03:39 PM Bug #12094 (Duplicate): "Segmentation fault" in smoke-master-distro-basic-multi run
Run: http://pulpito.ceph.com/teuthology-2015-06-19_05:00:05-smoke-master-distro-basic-multi/
Job: 940481
Logs: http...
Yuri Weinstein
11:03 AM Bug #12088 (Resolved): cephfs client crash after enable readahead mechanism through setting conf ...
I run fio tool to test the randread performance of cephfs. Ceph client will crash, when I enable readahead on cep... Yan Shen

06/18/2015

10:16 AM Bug #11989: Cephfs Kernel Client data corruption
Zheng Yan wrote:
> please try the attached patch
i have tried your patch with my test case with 300GB of data and...
Bernd Helm
03:52 AM Bug #11989: Cephfs Kernel Client data corruption
please try the attached patch Zheng Yan

06/17/2015

01:32 PM Bug #11989: Cephfs Kernel Client data corruption
I reproduced this locally Zheng Yan
09:24 AM Bug #11784: ceph-fuse hang on unmount (stuck dentry refs)
Hmm, there shouldn't have been any activity on the mount by this point. Maybe we've got some other kind of bug, though. Greg Farnum
12:58 AM Bug #11784: ceph-fuse hang on unmount (stuck dentry refs)
... Zheng Yan

06/16/2015

03:33 PM Bug #11758 (Pending Backport): kernel_untar_build fails on EL7
Greg Farnum
03:33 PM Bug #11758 (Resolved): kernel_untar_build fails on EL7
Greg Farnum
03:16 PM Bug #11758 (Fix Under Review): kernel_untar_build fails on EL7
https://github.com/ceph/ceph/pull/4967 Greg Farnum
01:52 PM Bug #11985 (In Progress): MDS asserts in objecter when transitioning from replay to DNE
John Spray
01:49 PM Bug #11541 (Resolved): MDS is crashed (mds/CDir.cc: 1391: FAILED assert(!is_complete()))
Zheng Yan
11:22 AM Bug #11913 (Resolved): Failure in TestClusterFull.test_barrier
commit:bf9a9a2d9ff2be129b303d535899f60ad49f7c23 Greg Farnum
10:23 AM Bug #12019: multiple_rsync failure
Yeah, I think this was me being silly when reading the log, I read straight from the rsync invocation to the error, w... John Spray
10:09 AM Bug #12019: multiple_rsync failure
Greg Farnum
10:09 AM Bug #12019: multiple_rsync failure
commit:0804655725d84d866a32826203638fcfd71d4b51
Since we're using sudo to copy we presumably need it to delete. I ...
Greg Farnum
07:13 AM Bug #12019 (Fix Under Review): multiple_rsync failure
https://github.com/ceph/ceph/pull/4964 Zheng Yan
07:10 AM Bug #11989: Cephfs Kernel Client data corruption
could you please provide me a list of corrupt blocks (offset and size of corrupt block). Besides, could you please tr... Zheng Yan

06/15/2015

09:56 AM Bug #12019 (Resolved): multiple_rsync failure
Related to #11781?
This is running with the recent change to source files from a dir in /tmp instead of directly f...
John Spray
05:50 AM Bug #11989: Cephfs Kernel Client data corruption
Zheng Yan wrote:
> are there any suspected message when this happens?
dmesg is silent, ceph logs in /var/log/ceph...
Bernd Helm
01:57 AM Bug #11989: Cephfs Kernel Client data corruption
are there any suspected message when this happens? Zheng Yan

06/12/2015

07:31 PM Backport #11999 (Resolved): cephfs Dumper tries to load whole journal into memory at once
https://github.com/ceph/ceph/pull/5120 Nathan Cutler
07:06 PM Bug #11989: Cephfs Kernel Client data corruption
I imagine this is a result of some kind of memory exhaustion, but I'm not sure how best to diagnose it or if there ar... Greg Farnum
01:22 PM Bug #11989 (Resolved): Cephfs Kernel Client data corruption
Hi. i get random data corruption with the cephfs kernel client. i do streaming from a non-ceph machine using "cat <fi... Bernd Helm
06:19 PM Bug #11986: logs changing during tarball generation at end of job
Actually this job doesn't have log rotation enabled at all. Greg Farnum
12:34 PM Bug #11986 (Closed): logs changing during tarball generation at end of job

http://pulpito.ceph.com/teuthology-2015-06-08_23:04:01-fs-master---basic-multi/926320/...
John Spray
12:31 PM Bug #11985 (Resolved): MDS asserts in objecter when transitioning from replay to DNE

Seen once:
http://pulpito.ceph.com/teuthology-2015-06-08_23:04:01-fs-master---basic-multi/926330/...
John Spray

06/11/2015

01:56 PM Feature #3826 (Resolved): uclient: Be more aggressive about checking for pools we can't write to
Zheng did this a few months ago; we now write to a test object in every newly-seen pool. Greg Farnum
01:55 PM Feature #4885 (Resolved): dumper: do an incremental log dump
https://github.com/ceph/ceph/pull/4835 Greg Farnum
01:52 PM Feature #11588 (Resolved): teuthology: set up log rotate for MDS logs
https://github.com/ceph/ceph-qa-suite/pull/452 Greg Farnum
01:28 PM Bug #11959 (Resolved): qa-suite: /usr copy needs more perms
Zheng Yan
10:13 AM Bug #11959 (Fix Under Review): qa-suite: /usr copy needs more perms
That's kind of yucky of virtual box to put those in /usr/lib but whatever!
https://github.com/ceph/ceph/pull/4930
John Spray
05:23 AM Bug #11959 (Resolved): qa-suite: /usr copy needs more perms
/a/ubuntu-2015-06-10_10:18:19-fs-greg-fs-testing---basic-multi/928341/teuthology.log... Greg Farnum
01:15 PM Bug #11913 (Fix Under Review): Failure in TestClusterFull.test_barrier
https://github.com/ceph/ceph-qa-suite/pull/457 John Spray

06/10/2015

08:34 PM Support #11923: MDS init script starts multiple instances when MDS is referenced in ceph.conf
Can you attach the ceph.conf and an ls -al of the instance directory? This will happen if the section is in ceph.co... Sage Weil
02:19 PM Bug #10950 (Resolved): Unable to remove MDS host: error handling
Merged to master in commit:5441f89c022aa1f4df084a4280e45c5c5b278f00 Greg Farnum
02:18 PM Bug #11746 (Pending Backport): cephfs Dumper tries to load whole journal into memory at once
Merged to master in commit:04a11f0f2f6d46091d6868ba1cc2fec7a4e7a99c Greg Farnum
01:19 PM Feature #11950 (Resolved): Strays enqueued for purge cause MDCache to exceed size limit

If your purge operations are going slowly (either because of throttle or because of slow data pool), and you do lot...
John Spray

06/08/2015

09:55 PM Support #11923 (Resolved): MDS init script starts multiple instances when MDS is referenced in ce...
MDS component init script does not seem to be able to properly differentiate between auto-detected instances and inst... Brian Andrus
08:35 PM Bug #6613: samba is crashing in teuthology
Seeing in release firefly v0.80.9 validation
Run: http://pulpito.ceph.com/ubuntu-2015-06-08_11:05:13-samba-firefly...
Yuri Weinstein
01:53 PM Bug #11913: Failure in TestClusterFull.test_barrier
(Referring to ceph-qa-suite/tasks/cephfs/test_full.py::test_barrier().)
So mount.a is doing open_no_data("alpha") ...
Greg Farnum
11:21 AM Bug #11913 (Resolved): Failure in TestClusterFull.test_barrier

http://pulpito.ceph.com/teuthology-2015-06-05_23:04:02-fs-master---basic-multi/922578/
mount_a is getting the ne...
John Spray

06/07/2015

07:45 AM Backport #11737 (In Progress): MDS is crashed (mds/CDir.cc: 1391: FAILED assert(!is_complete()))
Abhishek Lekshmanan

06/06/2015

09:52 AM Feature #11903 (Resolved): libcephfs: implement read/write interface support "struct iovec"
Now if qemu virtfs directly call libcephfs api, it need to retranslate iov structure to continuous buffer. We should ... Haomai Wang

06/05/2015

06:23 PM Bug #11807 (Resolved): workunits: stop rsyncing with /usr as the source material
3f4dfb80e32fba86e9076a2915197487bf899906
We can reopen if we decide this needs a backport.
Greg Farnum
06:14 PM Bug #11504 (Resolved): CephFS restriction on removing cache tiers is overly strict
commit:5a3da7b891bf18e0f164352dd0010c7969233d72 Greg Farnum

06/03/2015

10:51 AM Feature #11859 (Resolved): MDS "damage table" for recording scrub/fetch errors

Creating a ticket for this ongoing work in the interests of clarity.
The idea is that when we get an error in fe...
John Spray
09:42 AM Bug #10950 (Fix Under Review): Unable to remove MDS host: error handling
https://github.com/ceph/ceph/pull/4838 John Spray
09:19 AM Bug #11835 (Fix Under Review): FuseMount.umount_wait can hang
https://github.com/ceph/ceph-qa-suite/pull/453 John Spray
09:07 AM Bug #11746 (Fix Under Review): cephfs Dumper tries to load whole journal into memory at once
https://github.com/ceph/ceph/pull/4835 John Spray

06/02/2015

09:08 PM Bug #11758: kernel_untar_build fails on EL7
let's update the tarball to use a newer kernel. Sage Weil
05:55 PM Bug #11807 (Fix Under Review): workunits: stop rsyncing with /usr as the source material
https://github.com/ceph/ceph/pull/4831 John Spray
01:49 PM Bug #11807: workunits: stop rsyncing with /usr as the source material
John suggests we can grab /usr/doc instead, maybe. Greg Farnum
01:54 PM Bug #11300 (Resolved): client-limits and mds-full failures on clog warning/errors
We think we already did that...sorry, no commit refs handy. Greg Farnum
01:44 PM Bug #10950: Unable to remove MDS host: error handling
This is just unfriendly error handling. John says one of the other commands has a spurious argument that needs to get... Greg Farnum
01:42 PM Bug #11784: ceph-fuse hang on unmount (stuck dentry refs)
I copied the ceph-client log into that folder, although we're missing the server logs. Greg Farnum
01:36 PM Bug #11294: samba: DISCONNECTED inode warning
seems like dup of #11835 Zheng Yan

06/01/2015

09:55 PM Bug #11835 (Resolved): FuseMount.umount_wait can hang

Currently code in FuseMount.umount assumes that the write to /sys/fuse/connections/X/abort causes the process to te...
John Spray
09:49 PM Bug #11784: ceph-fuse hang on unmount (stuck dentry refs)
We saw this again today, and it's definitely inode refs this time:
http://pulpito-rdu.front.sepia.ceph.com/gregf-201...
Greg Farnum

05/31/2015

10:14 PM Bug #10248: messenger: failed Pipe;:connect::assert(m) in Hadoop client
Yes — the related issue chain has been seen a few times, more recently. Greg Farnum
06:05 PM Bug #10248: messenger: failed Pipe;:connect::assert(m) in Hadoop client
is it still valid ? Loïc Dachary

05/29/2015

06:15 PM Bug #11792: mds: recursive statistics are either inaccurate or too "chunky"
Well, they're delayed, yes. But the few times I've bothered to measure the delay is very short (a few seconds at most... Greg Farnum
07:13 AM Bug #11792: mds: recursive statistics are either inaccurate or too "chunky"
thanks to delay propagation, recursive statistics are never accurate Zheng Yan

05/28/2015

06:16 PM Bug #11781: multiple_rsync failure with fuse client
#11807. I'm not actually sure what good options there are for replacement, but it won't get lost. Greg Farnum
12:24 PM Bug #11781: multiple_rsync failure with fuse client
Hmm, maybe using /usr/ as a source of workload files is something we just need to stop doing, there's no particular r... John Spray
12:12 PM Bug #11781 (Rejected): multiple_rsync failure with fuse client
... Zheng Yan
06:15 PM Bug #11807 (Resolved): workunits: stop rsyncing with /usr as the source material
We've had various issues with /usr. The permissions on it are fiddly, its size can vary, and apparently some times th... Greg Farnum
11:47 AM Bug #11789: knfs mount fails with "getfh failed: Function not implemented"
I don't think this is likely to be a regression from previous hammer release, so not a blocker imho John Spray
06:17 AM Bug #11783 (In Progress): protocol: flushing caps on MDS restart can go bad
this is a message ordering issue when MDS failover.
chown marks Ax dirty
client flushes and releases Ax cap
cho...
Zheng Yan
01:02 AM Bug #11790 (Resolved): "cannot utime" errors in from tar in knfs workloads
it's nfs bug in 4.1-rc2 kernel. now the testing branch has been rebased to 4.1-rc5, this issue should disappear Zheng Yan

05/27/2015

10:07 PM Bug #11779 (Resolved): Intermittent failure in test_full
Greg Farnum
10:02 PM Bug #11779: Intermittent failure in test_full
The breaker was... John Spray
09:18 PM Bug #11779 (Pending Backport): Intermittent failure in test_full
Not sure when the change happened that broke this — do we need to backport to older qa branches? Greg Farnum
12:53 PM Bug #11779 (Fix Under Review): Intermittent failure in test_full
https://github.com/ceph/ceph-qa-suite/pull/447 John Spray
11:16 AM Bug #11779 (Resolved): Intermittent failure in test_full

It isn't waiting long enough before writing the last chunk of data. This used to work because the osd report inter...
John Spray
09:54 PM Bug #11781: multiple_rsync failure with fuse client
#9884 was the case where it timed out b/c there were just so many files, whereas in this instance it's actually faili... John Spray
09:22 PM Bug #11781: multiple_rsync failure with fuse client
Possibly a dup of #9884? :/ I'm not sure how accurate that "too many files" is as the actual bug cause, though... Greg Farnum
12:46 PM Bug #11781: multiple_rsync failure with fuse client
Wrong project, sorry. John Spray
12:28 PM Bug #11781 (Rejected): multiple_rsync failure with fuse client

The second rsync run is transferring some data, so either timestamps are going wrong, dentries are going missing, o...
John Spray
09:50 PM Bug #11790: "cannot utime" errors in from tar in knfs workloads
It's the failure to set mtimes etc on an extracted file via utimensat.
I have a slightly fuzzy memory that we were...
John Spray
09:25 PM Bug #11790: "cannot utime" errors in from tar in knfs workloads
... Greg Farnum
08:14 PM Bug #11790: "cannot utime" errors in from tar in knfs workloads
hammer v0.94.2 release
do you think is this a blocker?
Yuri Weinstein
07:21 PM Bug #11790 (Resolved): "cannot utime" errors in from tar in knfs workloads
http://pulpito.ceph.com/teuthology-2015-05-18_13:43:17-knfs-hammer-testing-basic-multi/897933/
http://pulpito.ceph.c...
John Spray
09:29 PM Bug #11301 (Resolved): mds-full test occasionally fails due to blacklist expiry
Greg Farnum
09:23 PM Bug #11783: protocol: flushing caps on MDS restart can go bad
Yep, this one looks unfamiliar to me. :( Do we have client logs from when it happened that we can reference? Greg Farnum
01:04 PM Bug #11783 (Resolved): protocol: flushing caps on MDS restart can go bad

Not consistent, not happening on master.
http://pulpito.ceph.com/teuthology-2015-05-16_23:04:02-fs-next-testing-...
John Spray
08:44 PM Bug #11792 (New): mds: recursive statistics are either inaccurate or too "chunky"
https://www.mail-archive.com/ceph-users@lists.ceph.com/msg20005.html... Greg Farnum
08:26 PM Bug #11784: ceph-fuse hang on unmount (stuck dentry refs)
John, is this likely to have been a dup of #11294? We can tell by checking out the ceph-fuse log if it's still availa... Greg Farnum
01:28 PM Bug #11784 (Can't reproduce): ceph-fuse hang on unmount (stuck dentry refs)
http://pulpito-rdu.front.sepia.ceph.com/teuthology-2015-05-19_23:14:02-samba-master-testing-basic-typica/21446/
Th...
John Spray
08:14 PM Bug #11789: knfs mount fails with "getfh failed: Function not implemented"
hammer v0.94.2 release
do you think is this a blocker?
Yuri Weinstein
07:08 PM Bug #11789 (Can't reproduce): knfs mount fails with "getfh failed: Function not implemented"

See plana37 syslog on /a/teuthology-2015-05-18_13:43:17-knfs-hammer-testing-basic-multi/897936...
John Spray
01:29 PM Bug #11756 (Resolved): 'ls /sys/fs/fuse/connections' causes fuse mount fails
John Spray

05/26/2015

10:36 AM Bug #11758 (Resolved): kernel_untar_build fails on EL7

Perhaps the more recent GCC is too strict to compile the old kernel tarball we use in the test?
http://magna002....
John Spray
03:12 AM Bug #11756: 'ls /sys/fs/fuse/connections' causes fuse mount fails
http://pulpito.ceph.com/teuthology-2015-05-22_23:04:02-fs-master-testing-basic-multi/906267/ Zheng Yan
03:01 AM Bug #11756 (Fix Under Review): 'ls /sys/fs/fuse/connections' causes fuse mount fails
https://github.com/ceph/ceph-qa-suite/pull/444 Zheng Yan
03:00 AM Bug #11756 (Resolved): 'ls /sys/fs/fuse/connections' causes fuse mount fails
before list fuse connections, we should make sure fusectl is mounted on /sys/fs/fuse/connections Zheng Yan
02:22 AM Bug #11752 (Resolved): valgrind InvalidRead
Zheng Yan

05/25/2015

03:56 AM Bug #11752 (Fix Under Review): valgrind InvalidRead
https://github.com/ceph/ceph/pull/4755 Zheng Yan
03:38 AM Bug #11752 (Resolved): valgrind InvalidRead
http://qa-proxy.ceph.com/teuthology/teuthology-2015-05-20_23:04:01-fs-master-testing-basic-multi/902897/teuthology.log Zheng Yan

05/22/2015

07:52 PM Bug #11746 (Resolved): cephfs Dumper tries to load whole journal into memory at once
It should be streaming instead.
(From http://lists.ceph.com/pipermail/ceph-users-ceph.com/2015-May/001622.html)
John Spray
01:14 PM Bug #11301 (Fix Under Review): mds-full test occasionally fails due to blacklist expiry
https://github.com/ceph/ceph/pull/4746
https://github.com/ceph/ceph-qa-suite/pull/443
John Spray
08:03 AM Backport #11737 (Resolved): MDS is crashed (mds/CDir.cc: 1391: FAILED assert(!is_complete()))
https://github.com/ceph/ceph/pull/4886 Loïc Dachary
 

Also available in: Atom