Bug #3601
client: With multiple clients, file remove doesn't free up space
0%
Description
If two or more clients have ceph mounted and one client removes a file, the space for that file doesn't get freed on the OSDs.
Reported by Geoffrey Hartz (http://comments.gmane.org/gmane.comp.file-systems.ceph.devel/11257).
The issue is that the client(s) that didn't remove the file hold the cap for that inode indefinitely. We will probably need a timeout on the dentry lru on the client, allowing files that haven't been accessed recently to have their caps flushed, so that if the file has been removed, the space for that file can be properly freed.
History
#1 Updated by Greg Farnum about 11 years ago
- Description updated (diff)
- Status changed from New to Duplicate
The LRU actually already exists; check out Client::lru. (Unless I'm misunderstanding something?) So we might want to add a timer to that. Or maybe we can come up with a non-crappy way to do cap revocation to prompt for it without requiring them to give up caps after x time on a file they access every x+1 time.
#2 Updated by Greg Farnum about 11 years ago
- Status changed from Duplicate to New
Whoops, didn't mean to change that status.
#3 Updated by Sam Lang about 11 years ago
Yeah its that the lru doesn't have a timeout.
The mds could send an "enable timeout" message to clients once it sees an unlink. That would only enable the timeout for caps on inodes that are likely to be removed.
#4 Updated by Michael Linke over 10 years ago
Is there any progress on this? I've tried CephFS with actual dev-version 0.66 (b6b48dbefadb39419f126d0e62c035e010906027)
on Ubuntu 12.04 LTS (Kernel 3.5.0-36-generic) [Client 1]
and on Ubuntu 13.04 (Kernel 3.8.0-26-generic) [Client 2]
without success. Do you think you will eliminate this bug until dumpling-release?
#5 Updated by Greg Farnum over 10 years ago
No progress yet; Inktank isn't able to devote much time to the filesystem right now.
See also #5039, which is the more general form of this specific issue.
#6 Updated by Greg Farnum over 10 years ago
- Assignee deleted (
Sam Lang)
#7 Updated by Zheng Yan over 10 years ago
I think we can try trimming the unlinked inode when receiving cap message with "nlink == 0".
#8 Updated by Sage Weil over 10 years ago
Zheng Yan wrote:
I think we can try trimming the unlinked inode when receiving cap message with "nlink == 0".
Yeah. Need to make sure mds sends those out even if the client doesn't have any wanted caps, and tweak the client to trim the inode in that case.
#9 Updated by Greg Farnum over 10 years ago
Just make sure to check that the client doesn't have the file opened locally before trimming. ;)
#10 Updated by Zheng Yan over 10 years ago
- File 0001-ceph-trim-deleted-inode.patch View added
- File 0001-mds-notify-clients-about-deleted-inode.patch View added
- File 0002-client-trim-deleted-inode.patch View added
#11 Updated by Sage Weil over 10 years ago
- Status changed from New to 7
repushing to wip-fuse for testing
#12 Updated by Zheng Yan over 10 years ago
- Status changed from 7 to Resolved
#13 Updated by Greg Farnum over 7 years ago
- Component(FS) Client added