Ceph » CephFS

CephFS (both ceph-fuse and kernel client) flush the content of files from page cache upon closing the file.

This behaviour is undesirable in case the same file has to be read by a subsequent process, which might be the case in certain
HPC workloads.

A patch was supplied that disabled the cache invalidation:

https://github.com/ceph/ceph/pull/6380
https://github.com/ceph/ceph-client/commit/dfbb503e4e12580fc3d2952269104f293b0ec7e8

fuse needs to be instructed to delegate cache invalidation to ceph-fuse using "fuse use invalidate cb" configuration setting
(default fuse behaviour is dropping content from cache upon open() call).

Some questions remain whether this patch might lead to dead locks:

"...as long as nobody blocks waiting for invalidate while holding a lock
(client_lock?) that could prevent other fuse ops like write (pretty sure
that was the deadlock we saw before). I worry this could still happen
with a writer (or reader?) getting stuck in a check_caps() type situation
while the invalidate cb is waiting on a page lock held by the calling
kernel syscall..."

Although simple tests on a single host with single interactions works well with the patch,
the correct behaviour in the mentioned scenario should be check prior to use this patch in
production environments.