Ceph » Linux kernel client

Background info:

http://tracker.ceph.com/projects/ceph/wiki/Inline_data_support_for_Ceph
http://tracker.ceph.com/projects/ceph/wiki/Mds_-_Inline_data_support_(Step_2)

I spent some time looking at this today, and I think I see a path forward. The current kclient code mostly tries to uninline the code as soon as possible during page_mkwrite or write_iter:

step 1: allow __send_cap to flush inline write data back to the MDS.Currently we fill out an args struct and then encode the thing from that. By that point though, we've dropped the i_ceph_lock, so I think we need to refactor the cap sending code to do the encoding while holding the i_ceph_lock. This code is shared by the capsnap encoding too, so that will also need to be changed. We'll only encode the inline data if write caps are dirty at the time. Otherwise we can skip it.

step 2: loosen up when we'll uninline the data. Do it closer to the point where we're going to be talking to the backing store. In the case where the file is less than a page in size at the time we're flushing, and it's not already uninlined, we can instead keep it inlined and just call try_flush_caps to flush it to the MDS.

Eventually, we may be able to handle larger inline sizes, maybe via a tunable of some sort.

I'm making some progress on this, but it has been rather slow going. I've made a couple of draft patchsets, but they've only served so far to convince me that they are not the way to go.

The existing code attempts to uninline things very early. What I think we need to do is turn that around write to the cache when we have correct caps, and only worry about where to write the data when we have to do writeback. That should theoretically be better for performance anyway -- applications won't need to synchronously wait for the uninlining to happen before they can write data.

For the case where we don't have buffer/lazyio caps, we'll just write through to the MDS by forcing a synchronous cap flush.

Many of these writeout situations are slightly different from one another, so the challenge has been to create infrastructure to handle this that is useful across a range of cases. At this point, I'm mostly trying to sort out what sort of infrastructure we'll need, and how to modify the existing code to use it.