Ceph » CephFS

Here's a reproducer for the incarnation issue:
https://github.com/ceph/ceph-qa-suite/tree/wip-15399

I note that when OSDs construct their objecters they just use the OSDMap epoch as the incarnation. I wonder why the MDS has a per-rank incarnation counter at all? Perhaps we can just remove it and use the MDSMap epoch instead.

• MDSes A and B come up during the same epoch
• A becomes active and B becomes standby
• A fails
• B starts replaying operations

Then B needs to have an incarnation which differs from A's. The OSDs are each a distinct daemon entity (unlike the MDSes).

There may be ways to simplify it, though!

So we could probably reset our network connections with an incarnation based on the last MDSMap where our role changed...I think that should work; maybe it's what you meant.

We only do objecter->set_client_incarnation(incarnation); in MDSRank::init (after we've been assigned an active role)

So epoch should be sufficient (it's always incremented when a rank assignment has changed) as long as we remember to set it when a standby replay MDS is promoted (as well as when MDSRank is initialized).

I think using the MDSMap epoch as the incarnation is good idea