Ceph » CephFS

Update the journaler interface to allow the MDS to 'tail' the journal... periodically check to see if it's been extended and read events as they are written. (Someday we can use the new watch/notify to make this efficient!)

The shadow mds will also need to 'trim' the expired part of the journal by periodically checking the jouranler header (and expire_pos) and trimming old LogSegments and associated metadata out of its cache. This will probably be tricky, but whoever does it will hopefully come out with a thorough grasp of how the replay works and can write some of it down!

I've been getting some proper time in on this on and off over the last few days. Pushed the Journaler changes to the branch standby_replay. Will be starting on making the MDS tail and update next!

Updated Journaler to make new interface options asynchronous.
Presently working on how to disambiguate between a one-shot and continuous replay (probably a new state) on the MDS and Monitor. Then implement basic continuous replay without worrying much about evicting stuff from the cache. Finally figure out how to effectively evict stuff from the cache without breaking interaction with the other MDSes.

I have yet to implement trimming, but the basic restarting-replay bits are now in place along with hooks to make it start. Testing is revealing a fair number of issues with the Journaler and MDLog, though -- they don't much like repeating this process!

Well, this seems to be working as best I can tell.

There are some odd issues with virtual memory usage growing by leaps and bounds, but heap analyzer tools (tcmalloc heapdump, massif) indicate that it's not actually using that much memory, so....fragmentation?
Yehuda and Sage can't come up with anything so we decided to table it unless we hear about real problems. Merged into unstable!

Bulk updating project=ceph category=mds bugs so that I can remove the MDS category from the Ceph project to avoid confusion.