Bug #15399
closed
MDS incarnation get lost after remove filesystem
Added by Zheng Yan about 8 years ago.
Updated almost 8 years ago.
Description
If we remove a filesystem, then create new filesystem with old data/metadata pools. OSD may drop requests from MDS of new filesystem, because OSD thinks the requests are duplicated.
A related issue is that MDS in different filesystems can have same client ID. This is a little weird
Here's a reproducer for the incarnation issue:
https://github.com/ceph/ceph-qa-suite/tree/wip-15399
I note that when OSDs construct their objecters they just use the OSDMap epoch as the incarnation. I wonder why the MDS has a per-rank incarnation counter at all? Perhaps we can just remove it and use the MDSMap epoch instead.
If:
- MDSes A and B come up during the same epoch
- A becomes active and B becomes standby
- A fails
- B starts replaying operations
Then B needs to have an incarnation which differs from A's. The OSDs are each a distinct daemon entity (unlike the MDSes).
There may be ways to simplify it, though!
So we could probably reset our network connections with an incarnation based on the last MDSMap where our role changed...I think that should work; maybe it's what you meant.
We only do objecter->set_client_incarnation(incarnation); in MDSRank::init (after we've been assigned an active role)
So epoch should be sufficient (it's always incremented when a rank assignment has changed) as long as we remember to set it when a standby replay MDS is promoted (as well as when MDSRank is initialized).
I think using the MDSMap epoch as the incarnation is good idea
- Status changed from New to In Progress
- Assignee set to John Spray
- Status changed from In Progress to Fix Under Review
- Status changed from Fix Under Review to Pending Backport
- Copied to Backport #15732: jewel: MDS incarnation get lost after remove filesystem added
- Status changed from Pending Backport to Resolved
Also available in: Atom
PDF