Project

General

Profile

Bug #11226

client: drops Fx cap on directories when revoking other caps

Added by Greg Farnum over 4 years ago. Updated about 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
03/24/2015
Due date:
% Done:

0%

Source:
Development
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Client
Labels (FS):
Pull request ID:

Description

An oddity was reported in the thread "MDS has inconsistent performance" on ceph-devel, starting at http://www.spinics.net/lists/ceph-devel/msg21987.html. The user noticed that when clients were doing a file create workload in their own directory, they would start out sending only creates to the MDS, but that eventually they switched to doing a prior lookup for every create. This of course was because the client was losing its I_COMPLETE flag on the directory.

Investigation revealed that I_COMPLETE was getting cleared because the client did a create while holding the Fs cap (but not Fx) on the directory, so it dropped the Fs cap as part of the create request, and then when it got re-assigned in the response it had to clear the flag. This was odd since the client had previously held Fsx on the directory.

The drop of Fx turned out to be the result of Client::check_caps(). check_caps() was invoked when the MDS revoked some unrelated caps, and check_caps() calls send_cap(), which drops all caps which aren't included in a "retain" parameter. check_caps is filling out retain with the combined output of Inode::caps_wanted() and the caps used on the inode.
Unfortunately, caps_wanted() only returns anything at all if the inode is opened, based on the mode in use. Directories are basically never opened, and their caps aren't used much (ever?) either.

Associated revisions

Revision 0c6bd277 (diff)
Added by Greg Farnum over 4 years ago

client: Hold on to exclusive caps on directories we "own"

If a directory is complete, we really want to keep the exclusive cap
so that we don't end up needing to do MDS lookup requests on every cache
miss.

Fixes: #11226

Signed-off-by: Greg Farnum <>

History

#1 Updated by Greg Farnum over 4 years ago

The naive solution is to try and retain the Fx cap on directories at all times, but that has implications for how we flush stuff out.

I do have a heuristic that I think works, though: retain Fx if the directory is complete. That lets us drop the cap if it won't be helpful, and whenever we're under cache pressure we'll have to flush out the directory's children first anyway.

#2 Updated by Greg Farnum over 4 years ago

  • Status changed from New to Need Review

#3 Updated by Greg Farnum over 4 years ago

  • Status changed from Need Review to Testing

Also more bugfixes around this from Zheng at https://github.com/ceph/ceph/pull/4177, in greg-fs-testing and queued up for a run right now!

#4 Updated by Greg Farnum over 4 years ago

Merged Zheng's patches to master in commit:413da564d4fd479954da7feb5b27b5d8fade1100, and mine in commit:7721b224c4b0d9bb9cb81e9683f226cb5af624f4.

#5 Updated by Greg Farnum over 4 years ago

  • Status changed from Testing to Resolved

#6 Updated by Greg Farnum about 3 years ago

  • Component(FS) Client added

Also available in: Atom PDF