Project

General

Profile

Bug #3601

client: With multiple clients, file remove doesn't free up space

Added by Sam Lang over 11 years ago. Updated over 7 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Development
Tags:
Backport:
Regression:
Severity:
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Client
Labels (FS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

If two or more clients have ceph mounted and one client removes a file, the space for that file doesn't get freed on the OSDs.

Reported by Geoffrey Hartz (http://comments.gmane.org/gmane.comp.file-systems.ceph.devel/11257).

The issue is that the client(s) that didn't remove the file hold the cap for that inode indefinitely. We will probably need a timeout on the dentry lru on the client, allowing files that haven't been accessed recently to have their caps flushed, so that if the file has been removed, the space for that file can be properly freed.

0001-ceph-trim-deleted-inode.patch View - kernel patch (2.43 KB) Zheng Yan, 07/20/2013 07:20 PM

0001-mds-notify-clients-about-deleted-inode.patch View - mds patch (1.03 KB) Zheng Yan, 07/20/2013 07:20 PM

0002-client-trim-deleted-inode.patch View - client patch (9 KB) Zheng Yan, 07/20/2013 07:20 PM

History

#1 Updated by Greg Farnum about 11 years ago

  • Description updated (diff)
  • Status changed from New to Duplicate

The LRU actually already exists; check out Client::lru. (Unless I'm misunderstanding something?) So we might want to add a timer to that. Or maybe we can come up with a non-crappy way to do cap revocation to prompt for it without requiring them to give up caps after x time on a file they access every x+1 time.

#2 Updated by Greg Farnum about 11 years ago

  • Status changed from Duplicate to New

Whoops, didn't mean to change that status.

#3 Updated by Sam Lang about 11 years ago

Yeah its that the lru doesn't have a timeout.

The mds could send an "enable timeout" message to clients once it sees an unlink. That would only enable the timeout for caps on inodes that are likely to be removed.

#4 Updated by Michael Linke over 10 years ago

Is there any progress on this? I've tried CephFS with actual dev-version 0.66 (b6b48dbefadb39419f126d0e62c035e010906027)
on Ubuntu 12.04 LTS (Kernel 3.5.0-36-generic) [Client 1]
and on Ubuntu 13.04 (Kernel 3.8.0-26-generic) [Client 2]
without success. Do you think you will eliminate this bug until dumpling-release?

#5 Updated by Greg Farnum over 10 years ago

No progress yet; Inktank isn't able to devote much time to the filesystem right now.
See also #5039, which is the more general form of this specific issue.

#6 Updated by Greg Farnum over 10 years ago

  • Assignee deleted (Sam Lang)

#7 Updated by Zheng Yan over 10 years ago

I think we can try trimming the unlinked inode when receiving cap message with "nlink == 0".

#8 Updated by Sage Weil over 10 years ago

Zheng Yan wrote:

I think we can try trimming the unlinked inode when receiving cap message with "nlink == 0".

Yeah. Need to make sure mds sends those out even if the client doesn't have any wanted caps, and tweak the client to trim the inode in that case.

#9 Updated by Greg Farnum over 10 years ago

Just make sure to check that the client doesn't have the file opened locally before trimming. ;)

#11 Updated by Sage Weil over 10 years ago

  • Status changed from New to 7

repushing to wip-fuse for testing

#12 Updated by Zheng Yan over 10 years ago

  • Status changed from 7 to Resolved

#13 Updated by Greg Farnum over 7 years ago

  • Component(FS) Client added

Also available in: Atom PDF