Project

General

Profile

Bug #6396

mds: recovery hits assert(!segment.empty()) when reissuing caps

Added by Greg Farnum almost 6 years ago. Updated about 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
09/25/2013
Due date:
% Done:

0%

Source:
Q/A
Tags:
Backport:
dumpling
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
MDS
Labels (FS):
Pull request ID:

Description

2013-09-21T03:51:23.207 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]: mds/MDLog.h: In function 'LogSegment* MDLog::get_current_segment()' thread 7f3773fbf700 time 2013-09-21 03:51:56.890761
2013-09-21T03:51:23.207 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]: mds/MDLog.h: 172: FAILED assert(!segments.empty())
2013-09-21T03:51:23.208 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]:  ceph version 0.69-172-g08a97ae (08a97ae45f4df58a6a8ea8a6400934d860cf5eb4)
2013-09-21T03:51:23.208 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]:  1: ceph-mds() [0x5683d4]
2013-09-21T03:51:23.208 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]:  2: (Locker::scatter_writebehind(ScatterLock*)+0x6d4) [0x6cb114]
2013-09-21T03:51:23.208 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]:  3: (Locker::simple_sync(SimpleLock*, bool*)+0x452) [0x6cbbc2]
2013-09-21T03:51:23.208 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]:  4: (Locker::file_eval(ScatterLock*, bool*)+0x597) [0x6cc3c7]
2013-09-21T03:51:23.208 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]:  5: (Locker::eval(CInode*, int, bool)+0x468) [0x6d6108]
2013-09-21T03:51:23.208 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]:  6: (MDCache::reissue_all_caps()+0x13c) [0x62e77c]
2013-09-21T03:51:23.209 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]:  7: (MDS::recovery_done()+0x8d) [0x5800ad]
2013-09-21T03:51:23.209 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]:  8: (MDS::handle_mds_map(MMDSMap*)+0x41a3) [0x590083]
2013-09-21T03:51:23.209 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]:  9: (MDS::handle_core_message(Message*)+0x5d3) [0x590d73]
2013-09-21T03:51:23.209 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]:  10: (MDS::_dispatch(Message*)+0x2f) [0x5914cf]
2013-09-21T03:51:23.209 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]:  11: (MDS::ms_dispatch(Message*)+0x1e3) [0x592fd3]
2013-09-21T03:51:23.210 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]:  12: (DispatchQueue::entry()+0x549) [0x96e599]
2013-09-21T03:51:23.210 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]:  13: (DispatchQueue::DispatchThread::entry()+0xd) [0x88f81d]
2013-09-21T03:51:23.210 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]:  14: (()+0x7e9a) [0x7f3777ac6e9a]
2013-09-21T03:51:23.210 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]:  15: (clone()+0x6d) [0x7f37762d0ccd]
2013-09-21T03:51:23.210 INFO:teuthology.task.ceph.mds.b-s-a.err:[10.214.132.24]:  NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

/a/teuthology-2013-09-20_23:01:15-fs-master-testing-basic-plana/8560

See also #6349 and #5458

History

#1 Updated by Zheng Yan almost 6 years ago

where to get the mds log? (I can access your lab now)

#2 Updated by Greg Farnum almost 6 years ago

/a/teuthology-2013-09-20_23:01:15-fs-master-testing-basic-plana/8560

All these archive dirs we reference here are on teuthology.front.sepia.ceph.com.

#3 Updated by Zheng Yan almost 6 years ago

By reading the mds log for #5458, I think these issues should be mostly fixed by commit b144170544(mds: properly return log replay error).

Also open a new pull request https://github.com/ceph/ceph/pull/679, which handles -ENOENT returned by the journaler.

#4 Updated by Zheng Yan almost 6 years ago

  • Status changed from New to Testing

#5 Updated by Sage Weil almost 6 years ago

  • Status changed from Testing to Resolved

yeah looks good!

#6 Updated by Greg Farnum almost 6 years ago

  • Status changed from Resolved to Pending Backport
  • Assignee set to Greg Farnum
  • Backport set to dumpling

This seems to be showing up in our nightlies a fair bit still; it should be backported.

#7 Updated by Greg Farnum almost 6 years ago

  • Status changed from Pending Backport to Resolved

Backported to dumpling as commit:cd1c3c9e00e90b19e83c1f11a48e516a7de93665

#8 Updated by Greg Farnum about 3 years ago

  • Component(FS) MDS added

Also available in: Atom PDF