limited commit_partial probably broke mds recovery
When I set up the MDS to limit the size of commits it sends to the OSDs, Sage points out that I probably broke safety since the commit machinery includes a header which contains the Dir version and the (completely up-to-date of in-memory state) parent pointer. This means that if the MDS crashed after sending one of the on-disk methods, but not all of them, then the on-disk version of the directory thinks it has more up-to-date data than it really does. Ouch!
#2 Updated by Sage Weil over 9 years ago
I'm afraid this still isn't right, because the header in the CDir object includes recursive stats that should be accurate for the items currently committed in the directory. We may need to independently track the committed vs current rstats for the dir in memory to make this work... :/
#3 Updated by Greg Farnum over 9 years ago
Hmmm. I was under the impression that inconsistencies like that would be taken care of during journal replay (ie, the MDS would notice that the directory hadn't been committed and would commit the entire thing). Is that wrong, or are you worried about stats getting out of sync in the case that we lose the journal?
#4 Updated by Greg Farnum over 9 years ago
Okay, after discussing this with Sage he's happy -- if we lose the journal there's not a lot we can do about partial moves splitting files across directories (regardless of how the moves are committed), and the rstats will be recalculated anyway as part of the fsck.
Meanwhile, if we don't lose the journal, all this stuff will be taken care of safely as long as the on-disk version isn't a lie. And thanks to this commit, it's not!
Pushed to master in dc8ff94ee74b909f677174116a594baa7539aefb