Ceph

Narrowed this down through Joao's comments and bisecting to hit this commit. Not sure if this only happens under specific circumstances (Perhaps due to the way CBT is creating OSDs?) It appears to be very reproducible on all branches that include this commit up to and including the latest giant and master builds as of 10/21/2014. anything prior to this commit (including the v0.86 release) appears to work perfectly.

Joao's comments after examining the mon logs:

18:17 < joao> ah, the osd is seen initially by the monitor as a 'client.'
18:17 < joao> no wonder adding caps to 'osd.X' doesn't help
18:17 < joao> now I wonder why this happens
18:23 < joao> nhm, the only feasible explanation is that we used to serve maps
to everyone regardless of caps
18:23 < joao> which stopped being so after my patch
18:24 < nhm> hrm, are the caps wrong?
18:26 < joao> no, and what's weird is that the patch has been merged for a
while and this is the first time I'm hearing about this

I believe Joao is referring to c0e3bc9a above, though it does not (at least in isolation) appear to be the culprit.