Project

General

Profile

Activity

From 06/13/2011 to 07/12/2011

07/10/2011

02:22 PM Bug #1295 (Resolved): mds: handle rmdir of subtree root
If we rmdir an empty directory whose dirfrag(s) are exports/subtree roots,
- it gets journaled locally, replay is...
Sage Weil

07/08/2011

02:26 PM Bug #1167: Fsstress: bad rstat assert
merged into master in commit:73d98a057bfeaf6f9a87c38f85b199c6fdbbbf04 Greg Farnum
02:21 PM Bug #1167 (Resolved): Fsstress: bad rstat assert
Sage Weil

07/07/2011

01:11 PM Bug #1289 (Resolved): misdirected mds request results in improper ENOENT during mds rename workunits
Resolved by commit:3ef1e656e95eacb1d0d8acd530e2cd9cbe08d586.
We had some problems with false ENOENTs on lookups th...
Greg Farnum
11:24 AM Bug #1289 (Resolved): misdirected mds request results in improper ENOENT during mds rename workunits
Detected as #1288, the kclient is doing a lookup on mds0 for "srci2" immediately following a create "srci2" request. ... Greg Farnum

07/06/2011

04:19 PM Bug #1284 (Can't reproduce): client: leaked inode ref with cfuse_workunit_snaps.yaml
Seems to be pretty rare. ... Sage Weil

07/05/2011

02:40 PM Feature #1276 (Resolved): client: expose mds partition via virtual xattrs
our options are basically and ioctl or virtual xattrs. virtual xattrs are easier to use and portable across fuse and... Sage Weil
02:36 PM Bug #1252 (Resolved): regression: cfuse fails snaptest-2 with 2-MDS system
commit:e9e3883d0db4cc03183c0432773825f84170c42b Sage Weil

07/01/2011

10:15 AM Bug #1252 (Resolved): regression: cfuse fails snaptest-2 with 2-MDS system
On head of master, it asserts: ... Greg Farnum
09:00 AM Bug #1239 (Resolved): client: hang on umount after snaps workunit
Sage Weil

06/29/2011

03:32 PM Bug #1243 (Resolved): inest lock blocks dir create for a long time
commit:310032ee8128f6417ac302c0f5ecd27c691cbcc7
I haven't been able to figure out why this doesn't impact cfuse, t...
Greg Farnum
01:43 PM Bug #1243 (Resolved): inest lock blocks dir create for a long time
From the mailing list:
steps are:
we mount ceph on /mnt/test/
then create dir "/mnt/test/a/b/"
1) in dir "b" , us...
Greg Farnum
09:35 AM Bug #1232 (Closed): client: inode ref leak?
Sage Weil
09:34 AM Bug #1239 (Resolved): client: hang on umount after snaps workunit
two inodes still have pins. looks like a ref count leak somewhere.... Sage Weil
09:11 AM Bug #1238 (Resolved): mds: ~elist crash on untar_snap_rm
Sage Weil

06/28/2011

03:09 PM Bug #1193 (Resolved): cmds crashes with stacktrace when using --reset-journal and an invalid rank
Should be fixed by a pair of commits ending in commit:d1fcffad6c038c6712f1d83735e629017829a2fb Greg Farnum
02:11 PM Bug #1238 (Resolved): mds: ~elist crash on untar_snap_rm
INFO:teuthology.task.workunit.client.0.out:removed `k/coreutils-8.5/lib/stripslash.c'
INFO:teuthology.task.workunit....
Sage Weil
12:28 PM Feature #1237 (Resolved): mds caps limit mount to some subdir
About a year ago the ceph.conf supported something like:... Wido den Hollander
10:51 AM Feature #1236 (Resolved): libceph: set layout via virtual xattrs (libceph/cfuse)
Apparently there actually is support in cfuse for ioctls, although it's hairy. Given that, we should support them in ... Greg Farnum

06/27/2011

09:23 AM Bug #1225 (Resolved): client: recallstate removing caps when it shouldn't
Sage Weil

06/26/2011

10:33 PM Bug #1232 (Closed): client: inode ref leak?
see ginormous cfuse log on vapre Sage Weil

06/24/2011

11:16 AM Bug #1225 (Resolved): client: recallstate removing caps when it shouldn't
- can't have an inode with no caps
- cap lru needs to align with dentry lru to get reasonable results
Sage Weil

06/22/2011

10:00 PM Bug #1206: NFS reexport file creation lags 1-3 seconds
0.3 seconds for kernel mount... 0.01s for cfuse.. I'm going to work on tracking this down.. Brian Chrisman
09:05 PM Bug #1206: NFS reexport file creation lags 1-3 seconds
ok, but are you saying that a native mount is also .5s for touch? that's no good either! Sage Weil
04:59 PM Bug #1206: NFS reexport file creation lags 1-3 seconds
Ahh I don't seem to have access to close it... Brian Chrisman
04:58 PM Bug #1206: NFS reexport file creation lags 1-3 seconds
Ahh okay... file create isn't really any slower than on native ceph kernel mount (rather than re-export)... so this i... Brian Chrisman
03:42 PM Bug #1206: NFS reexport file creation lags 1-3 seconds
That's faster... 0.5s for a touch create.
As a reference local filesystem and another cfs exported over nfs runs sub...
Brian Chrisman
03:14 PM Bug #1206: NFS reexport file creation lags 1-3 seconds
It looks like after the mknod nfsd is calling write_inode via commit_metadata() in fs/nfsd/vfs.c. This triggers cap ... Sage Weil
04:39 PM Bug #1199 (Resolved): 4-MDS fsstress: remote ino lookup assert
Pushed to master in commit:bc273957af2224d028626a961fdb1bf3bc153f40 and commit:e8b45732a1e77bf4811335ae7b8c97c72cd342ce. Greg Farnum
04:37 PM Bug #1178 (Resolved): cfuse doesn't shut down -- Fw cap doesn't flush on one inode
Pushed to master in commit:787f03cbd7489cb31f5feeec08f84d0b68aa5c0a. Greg Farnum

06/21/2011

03:53 PM Bug #1195: ceph mds crash on version upgrade
pushed fix commit:cc644b842261dbeefde804ed999061b8733a9190 to stable branch Sage Weil
09:21 AM Bug #1110: mds: ls -l hangs on concurrent writer

> - when the client sends caps back to the MDS, some of them are dirty. The MDS, as a rule, will always make sure...
Andre Noll

06/20/2011

05:09 PM Bug #1206: NFS reexport file creation lags 1-3 seconds
If I reexport the ceph filesystem via fuse/nfs, there is *no* lag in creating files. Brian Chrisman
05:02 PM Bug #1206 (Closed): NFS reexport file creation lags 1-3 seconds
I'm attaching the kernel logs and mds logs for the creation of a file called scale-product/testfoo3 via a touch runni... Brian Chrisman
04:29 PM Bug #1199: 4-MDS fsstress: remote ino lookup assert
The problem is that if the backtrace for a given inode changed, the AnchorServer didn't overwrite the old Anchor that... Greg Farnum
03:31 PM Bug #1199: 4-MDS fsstress: remote ino lookup assert
Okay, this seems to be an anchortable bug. It's getting the last trace that got committed for the inode in question, ... Greg Farnum
04:28 PM Bug #1200 (Duplicate): 4-MDS fsstress remote ino lookup cycle
Yep, this is the same as #1199. Greg Farnum
03:15 PM Bug #1110 (Resolved): mds: ls -l hangs on concurrent writer
Created a separate task for the request prioritiziation on the osd, #1203 Sage Weil
09:22 AM Bug #1110: mds: ls -l hangs on concurrent writer
Okay, so I found one other issue while testing this patch. So there are two reasons this is slow:
- when we revo...
Sage Weil
08:18 AM Bug #1110: mds: ls -l hangs on concurrent writer
> Indeed: It was at epoch 5 which has a "modified" timestamp of 2011-06-15 22:37:48. This matches exactly the time wh... Andre Noll

06/17/2011

03:30 PM Bug #1200: 4-MDS fsstress remote ino lookup cycle
Looks like this is the same root cause as #1199. Greg Farnum
03:19 PM Bug #1200 (Duplicate): 4-MDS fsstress remote ino lookup cycle
Notice how it's missing an expected ino in a dir that's marked complete, and then it just tries to do the lookup agan... Greg Farnum
03:12 PM Bug #1199 (Resolved): 4-MDS fsstress: remote ino lookup assert
... Greg Farnum
11:04 AM Bug #1178: cfuse doesn't shut down -- Fw cap doesn't flush on one inode
Will be testing locally, push later. We had to make sure to check the mseq when setting a new auth_cap based on the M... Greg Farnum
09:41 AM Bug #1195: ceph mds crash on version upgrade
Oh, you have multiple MDSs.. can you dump the same object for whichever one(s) crashed in
13: (SessionMap::decode...
Sage Weil
09:38 AM Bug #1195: ceph mds crash on version upgrade
Attached log from mds crash with suggested debugging enabled. Sam Lang
09:30 AM Bug #1195: ceph mds crash on version upgrade
Attached result of above command. Sam Lang
09:25 AM Bug #1195: ceph mds crash on version upgrade
Can you 'rados -p metadata get mds0_sessionmap /tmp/mds0_sessionmap' and attach? I'm curious what is in the object t... Sage Weil
09:14 AM Bug #1195: ceph mds crash on version upgrade
Hmm, Ceph *should* be upgradable to newer versions. These backtraces don't look familiar though and i don't see anyth... Greg Farnum
08:03 AM Bug #1195 (Resolved): ceph mds crash on version upgrade
This may just not be something that's handled in Ceph yet, but while trying to upgrade from stable v0.28 to v0.29, I ... Sam Lang
08:59 AM Bug #1110: mds: ls -l hangs on concurrent writer
Sage Weil wrote:
> Can you confirm that it hung when the fs was full? Because that's normal (and unrelated to this ...
Andre Noll

06/16/2011

03:27 PM Bug #1178: cfuse doesn't shut down -- Fw cap doesn't flush on one inode
I'm pretty sure the client is getting the right import/export messages, there's just something going wrong in the twi... Greg Farnum
02:56 PM Bug #1178: cfuse doesn't shut down -- Fw cap doesn't flush on one inode
oh, interesting. in that case it won't get an IMPORT event because it wasn't exported.. and the client doesn't find ... Sage Weil
02:48 PM Bug #1178: cfuse doesn't shut down -- Fw cap doesn't flush on one inode
Oh, so it's an inode where the auth got changed, and it only has the cap for the new auth but it doesn't think it is ... Greg Farnum
01:48 PM Bug #1178 (In Progress): cfuse doesn't shut down -- Fw cap doesn't flush on one inode
Looking at this now. Greg Farnum
02:11 PM Bug #1114: NFS export extreme slowdown
I think I've figured out the more general symptom.
File creation via NFS-ontop-of-ceph takes 1-3 seconds per file.
...
Brian Chrisman
11:29 AM Bug #1110: mds: ls -l hangs on concurrent writer
Can you confirm that it hung when the fs was full? Because that's normal (and unrelated to this issue); if the FS is... Sage Weil
01:42 AM Bug #1110: mds: ls -l hangs on concurrent writer
> > Otherwise, there's nothing there. If the situation is reproducible, the best would be an mds log generated with ... Andre Noll
09:39 AM Bug #1193 (Resolved): cmds crashes with stacktrace when using --reset-journal and an invalid rank
Here are the two stack traces I noticed that are different.... Damien Churchill
07:56 AM Bug #1170: 2-MDS fsstress: SR fails subtree assert
I'm seeing this with the 0.29 packages, using Ubuntu Natty and Linux 3.0rc3 built from git. Only some of the crashed ... Damien Churchill

06/15/2011

08:06 PM Bug #1158: Unfinished freeze hangs fsstress
Unfortunately, adjust_nested_auth_pins never sees the person who actually grabbed it. The others print out the grabbi... Greg Farnum
06:51 PM Bug #1158: Unfinished freeze hangs fsstress
If you can reproduce, you can enable the auth pin set define in mdstypes.h, which tracks who the pinners are.
//#d...
Sage Weil
06:04 PM Bug #1158: Unfinished freeze hangs fsstress
Well, it's a nested auth pin.... Greg Farnum
04:32 PM Bug #1114: NFS export extreme slowdown
Here's the time sequence with each x-axis increment being 20 seconds.
Plotted separately to avoid scaling axes and a...
Brian Chrisman
04:16 PM Bug #1114: NFS export extreme slowdown
Here's a 20-second interval time evolution of /sys/kernel/debug/ceph/*/bdi/stats
I can go ahead and graph these..
Brian Chrisman
06:54 AM Bug #1110: mds: ls -l hangs on concurrent writer
Sage Weil wrote:
> I do see this:
> 2011-06-10 11:41:04.325925 7f931b987700 mds0.objecter FULL, paused modify 0x7f...
Andre Noll

06/14/2011

12:25 PM Bug #1110: mds: ls -l hangs on concurrent writer
Oh! Sorry, I didn't scroll down. :)
I do see this:
2011-06-10 11:41:04.325925 7f931b987700 mds0.objecter FULL,...
Sage Weil
10:08 AM Bug #1110: mds: ls -l hangs on concurrent writer
Sage Weil wrote:
> Oh, that log is from the 8th, and doesn't include the fix I pushed on the 9th fixing that particu...
Andre Noll
09:39 AM Bug #1110: mds: ls -l hangs on concurrent writer
Oh, that log is from the 8th, and doesn't include the fix I pushed on the 9th fixing that particular bug. Did you at... Sage Weil
01:41 AM Bug #1110: mds: ls -l hangs on concurrent writer
Sage Weil wrote:
> Andre, any chance you can pass along more information about the hang you were seeing? Log, or th...
Andre Noll

06/13/2011

10:21 PM Bug #1137 (Can't reproduce): MDS Crash
If this turns up again, let us know! I suspect it may be related to the rename journaling changes; I'll be testing f... Sage Weil
10:19 PM Bug #1110: mds: ls -l hangs on concurrent writer
Andre, any chance you can pass along more information about the hang you were seeing? Log, or the specific workload? Sage Weil
08:57 PM Bug #1084: blogbench won't finish: waiting for Fr cap forever
It is working great. I haven't see this problem after applying the patches. Henry Chang
09:40 AM Bug #1084 (Resolved): blogbench won't finish: waiting for Fr cap forever
I can't reproduce this on my mds_issue_caps, so I've merged it into master, commit:0f8fbd562cc58351541ac4f60a1dc98bdd... Sage Weil
12:00 PM Bug #1183 (Resolved): mds: snaptest-snap-rm-cmp.sh failing
osd regression, fixed by commit:056a0c5a008fef0924b17c7e24c85cf73b3b390e Sage Weil
11:03 AM Bug #1183 (Resolved): mds: snaptest-snap-rm-cmp.sh failing
this used to work :( Sage Weil
10:07 AM Bug #1181 (Can't reproduce): mds: old_inodes crash
... Sage Weil
 

Also available in: Atom