Ceph » RADOS

Because OSDs are now normal clients of each other in some circumstances, we've broken our map synchronization guarantees:
OSD1.objecter(10) -- osd_op_request() --> OSD2
OSD1.objecter(10) <-- osd_op_reply() -- OSD2 // note that the objecter doesn't need to get updated to take the reply
OSD1 <- payload -- objecter(10) // note that the OSD and Objecter don't share reply epochs

This is okay for most of the stuff we do (basic copy-get), but can cause problems when doing promotes that race against snap deletion in the map and such.