Ceph » CephFS

I have a dir that I used to snapshot before I switched to hardlink trees. I create files and subdirs in it, then, when I want to take a snapshot, I move all entries in it to a newly-created dir.

mds crashes as I move each entry out of it. Restarting the mds, it recovers successfully and completes the move of each entry.

I found out the problem is in cow. add_remote_dentry tries to create an entry for the about-to-be-moved file or subdir, but an entry already exists, presumably because I moved the same entries out previously, and I didn't create any newer .snap snapshots on the system, so the id it tries to use remains the same => bug.

Now, I don't think those entries should have been there in the first place; the dir isn't snapshotted, so the cows are wasteful, and they're not being cleaned up even if they're being created just in case, and they surely conflict with other versions of the directory created afterwards.

Plus, I don't understand why the replay should succeed: oddly, the items map in the dir starts empty and new entries are put in as each move is replayed; only later, during rejoin, are older entries _fetched. This seems odd.

Anyway, here's a rough sequence of steps that should suffice to trigger the problem, with the caveat that it might be necessary to ensure some steps are fully committed to the osds and pushed out of the mds journal in order for the problem to trigger, and no other snapshotting action should take place in the filesystem:

mkdir foo save
mkdir foo/.snap/whatever
rmdir foo/.snap/whatever
mkdir foo/bar
copy /stuff into foo/bar
mkdir save/1
mv foo/bar save/1
mkdir foo/bar
copy /stuff into foo/bar
mkdir save/2
mv foo/bar save/2
mkdir foo/bar
copy /stuff into foo/bar
mkdir save/3
mv foo/bar save/3