Ceph

Jim Schutt has reported some troubling behavior apparently involved with client caps surrounding distributed (but not simultaneous) use of files. See the "Odd "data used" reporting behavior by ceph -w" thread (http://www.spinics.net/lists/ceph-devel/msg01222.html).

After some effort I was able to reproduce his issue with dd-truncate-dd in an environment using vstart.sh (1 of each daemon), the kernel client (via UML) doing the dd, and cfuse doing the truncate. This did not occur if cfuse was used to perform the dd and the kernel client performed the truncate. I eventually discovered an issue with loner assignment in the MDS that made the kclient-based dd succeed, although obviously this loner assignment could not be the sole cause.

Jim reported back that this didn't seem to alleviate his problem at all. Now we need to fix it.