Bug #20313: Assertion in handle_dir_update - CephFS - Ceph

Actions

Copy link

Bug #20313

closed

Assertion in handle_dir_update

Added by John Spray almost 7 years ago. Updated about 5 years ago.

Status:

Resolved

Priority:

Urgent

Assignee:

Zheng Yan

Category:

Target version:

% Done:

Source:

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Component(FS):

Labels (FS):

multimds

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

Seen in test branch that had the following PRs in it:
[15125] mds: miscellaneous multimds fixes part2
[15510] mds: miscellaneous multimds fixes part3
[15154] [DNM] Remove "experimental" warnings from multimds
[15411] qa: misc cephfs test improvements
[15430] osdc: always use finisher from wait_for_readable
[15440] mds: multimds flock fixes

Unsure if the issue already existed in master or was a side effect of something else.

It originally looked like a hung kclient unmount -- I logged in and killed the umount command and restarted the crashed MDS (mds.a) so that the test would complete and gather the logs for us.

e/12.0.3-1684-gaab25ab/rpm/el7/BUILD/ceph-12.0.3-1684-gaab25ab/src/mds/MDCache.cc: 10397: FAILED assert(r == 0)
2017-06-15T03:55:44.151 INFO:tasks.ceph.mds.a.smithi141.stderr: ceph version 12.0.3-1684-gaab25ab (aab25abdb8cbdf67be2904c7f539c0c15990395f) luminous (dev)
2017-06-15T03:55:44.152 INFO:tasks.ceph.mds.a.smithi141.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x110) [0x55fa403c87b0]
2017-06-15T03:55:44.152 INFO:tasks.ceph.mds.a.smithi141.stderr: 2: (MDCache::handle_dir_update(MDirUpdate*)+0x65a) [0x55fa401a184a]
2017-06-15T03:55:44.152 INFO:tasks.ceph.mds.a.smithi141.stderr: 3: (MDCache::dispatch(Message*)+0xd5) [0x55fa401caf05]
2017-06-15T03:55:44.152 INFO:tasks.ceph.mds.a.smithi141.stderr: 4: (MDSRank::handle_deferrable_message(Message*)+0x5bc) [0x55fa400b53ec]
2017-06-15T03:55:44.152 INFO:tasks.ceph.mds.a.smithi141.stderr: 5: (MDSRank::_dispatch(Message*, bool)+0x1eb) [0x55fa400c2b0b]
2017-06-15T03:55:44.152 INFO:tasks.ceph.mds.a.smithi141.stderr: 6: (MDSRank::retry_dispatch(Message*)+0x12) [0x55fa400c3a72]
2017-06-15T03:55:44.152 INFO:tasks.ceph.mds.a.smithi141.stderr: 7: (MDSInternalContextBase::complete(int)+0x1eb) [0x55fa403142db]
2017-06-15T03:55:44.152 INFO:tasks.ceph.mds.a.smithi141.stderr: 8: (MDSRank::_advance_queues()+0x4a5) [0x55fa400c2345]
2017-06-15T03:55:44.153 INFO:tasks.ceph.mds.a.smithi141.stderr: 9: (MDSRank::_dispatch(Message*, bool)+0x4bd) [0x55fa400c2ddd]
2017-06-15T03:55:44.153 INFO:tasks.ceph.mds.a.smithi141.stderr: 10: (MDSRankDispatcher::ms_dispatch(Message*)+0x15) [0x55fa400c3a55]
2017-06-15T03:55:44.153 INFO:tasks.ceph.mds.a.smithi141.stderr: 11: (MDSDaemon::ms_dispatch(Message*)+0xf3) [0x55fa400ad283]
2017-06-15T03:55:44.153 INFO:tasks.ceph.mds.a.smithi141.stderr: 12: (DispatchQueue::entry()+0x792) [0x55fa40625732]
2017-06-15T03:55:44.153 INFO:tasks.ceph.mds.a.smithi141.stderr: 13: (DispatchQueue::DispatchThread::entry()+0xd) [0x55fa404444ed]
2017-06-15T03:55:44.153 INFO:tasks.ceph.mds.a.smithi141.stderr: 14: (()+0x7dc5) [0x7f04bb38edc5]
2017-06-15T03:55:44.153 INFO:tasks.ceph.mds.a.smithi141.stderr: 15: (clone()+0x6d) [0x7f04ba47373d]

http://pulpito.ceph.com/jspray-2017-06-15_02:50:24-multimds-wip-jcsp-testing-20170614-testing-basic-smithi/1288094