Ceph

wip-6605, pull request 787, e02740ac5da7c9f5e4c1fdd603918e56c05123de

Greg Farnum wrote:

It turns out that "osd rm" does not eliminate the osd's auth keys from the monitor. That's liable to cause issues. Correct the oversight and check for any other osd state that might be left around.

This was not fixed. Fixing this would need us to be able to seamlessly pack a change to the AuthMonitor state into the OSDMonitor's paxos proposal. Although this would be possible with a little bit of an effort, it would be a new feature altogether rather than a bug fix. Proposing both changes at different times isn't an option either as we would lose the expected atomicity. Instead, we delegate to the user, as we currently do, the responsibility of removing the osd key from the keyring upon osd removal.

Greg Farnum wrote:

This is particularly unfortunate because it looks like preprocess_boot will not block "old" instances of the OSD from joining and booting the "new" one out. :/

This was indeed a bug, however unrelated to the above. This behavior is fixed by wip-6605 and we will now let an osd boot given the following circumstances:

• osd exists and it's fsid is the same we have on record (in the osdmap)
• osd dne and fsid on record is nil

merged into master