Bug #10816: "mds e5 Missing health data for MDS 4117" endless loop in upgrade:firefly-x-hammer-distro-basic-multi run - teuthology - Ceph

Actions

Copy link

Bug #10816

closed

"mds e5 Missing health data for MDS 4117" endless loop in upgrade:firefly-x-hammer-distro-basic-multi run

Added by Yuri Weinstein over 9 years ago. Updated about 9 years ago.

Status:

Resolved

Priority:

Urgent

Assignee:

Yuri Weinstein

Category:

% Done:

Source:

Q/A

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Crash signature (v1):

Crash signature (v2):

Description

Run: http://pulpito.ceph.com/teuthology-2015-02-08_17:13:01-upgrade:firefly-x-hammer-distro-basic-multi/
Job: 744272
Logs: http://qa-proxy.ceph.com/teuthology/teuthology-2015-02-08_17:13:01-upgrade:firefly-x-hammer-distro-basic-multi/744272/

2015-02-08T18:13:43.812 INFO:tasks.ceph.mon.a.plana89.stderr:2015-02-08 18:13:43.811577 7f81a210c700 -1 mon.a@0(leader).mds e5 Missing health data for MDS 4117

Also see similar in run:
http://pulpito.front.sepia.ceph.com/teuthology-2015-02-07_17:18:02-upgrade:firefly-x-hammer-distro-basic-vps/
Job: 743728
Logs: http://qa-proxy.ceph.com/teuthology/teuthology-2015-02-07_17:18:02-upgrade:firefly-x-hammer-distro-basic-vps/743728/

Actions

Copy link

Updated by Greg Farnum over 9 years ago

Project changed from Ceph to teuthology
Assignee set to Yuri Weinstein

At a guess, you've upgraded the monitors and the MDS has not been upgraded yet. (Or some but not all of the monitors?)

This is usually going to be seen when upgrading and just means that somebody between the MDS and the leader (inclusive) isn't generating or propagating health data. It should be whitelisted.

Actions

Copy link

Updated by Yuri Weinstein over 9 years ago

Assignee changed from Yuri Weinstein to Loïc Dachary

Loic - it's coming from https://github.com/ceph/ceph-qa-suite/tree/hammer/suites/upgrade/firefly-x/stress-split-erasure-code
Can you pls take a look?

This suite was passing not too long ago - http://pulpito.front.sepia.ceph.com/teuthology-2015-01-24_17:13:01-upgrade:firefly-x-next-distro-basic-multi/

Actions

Copy link

Updated by Loïc Dachary over 9 years ago

Assignee changed from Loïc Dachary to Yuri Weinstein

I can't think of a reason why erasure code would cause failure in this situation. And even if it did, I can't hink of a way for an erasure code failure to manifest itself in this way. In addition erasure code has not changed recently.

Actions

Copy link