Project

General

Profile

Actions

Bug #36350

closed

mds: "src/mds/MDLog.cc: 281: FAILED ceph_assert(!capped)" during max_mds thrashing

Added by Patrick Donnelly over 5 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Immediate
Assignee:
Category:
-
Target version:
% Done:

0%

Source:
Q/A
Tags:
Backport:
mimic,luminous
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
MDS
Labels (FS):
crash
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2018-10-06T23:06:23.135 INFO:tasks.ceph.mds.a.smithi023.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.0.0-3907-g276c86e/rpm/el7/BUILD/ceph-14.0.0-3907-g276c86e/src/mds/MDLog.cc: In function 'void MDLog::_submit_entry(LogEvent*, MDSLogContextBase*)' thread 7f2734577700 time 2018-10-06 23:06:23.139488
2018-10-06T23:06:23.136 INFO:tasks.ceph.mds.a.smithi023.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.0.0-3907-g276c86e/rpm/el7/BUILD/ceph-14.0.0-3907-g276c86e/src/mds/MDLog.cc: 281: FAILED ceph_assert(!capped)
2018-10-06T23:06:23.136 INFO:tasks.ceph.mds.a.smithi023.stderr: ceph version 14.0.0-3907-g276c86e (276c86e4890a25e5e74b3f30dbb94987ede03b5a) nautilus (dev)
2018-10-06T23:06:23.137 INFO:tasks.ceph.mds.a.smithi023.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x153) [0x7f274142d829]
2018-10-06T23:06:23.137 INFO:tasks.ceph.mds.a.smithi023.stderr: 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x7f274142d9f0]
2018-10-06T23:06:23.137 INFO:tasks.ceph.mds.a.smithi023.stderr: 3: (MDLog::_submit_entry(LogEvent*, MDSLogContextBase*)+0x591) [0x557160cd6a31]
2018-10-06T23:06:23.137 INFO:tasks.ceph.mds.a.smithi023.stderr: 4: (Locker::scatter_writebehind(ScatterLock*)+0x2b4) [0x557160bbb6b4]
2018-10-06T23:06:23.137 INFO:tasks.ceph.mds.a.smithi023.stderr: 5: (Locker::simple_lock(SimpleLock*, bool*)+0x320) [0x557160bbcd30]
2018-10-06T23:06:23.137 INFO:tasks.ceph.mds.a.smithi023.stderr: 6: (Locker::scatter_nudge(ScatterLock*, MDSInternalContextBase*, bool)+0x4a3) [0x557160bc03c3]
2018-10-06T23:06:23.137 INFO:tasks.ceph.mds.a.smithi023.stderr: 7: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase<MDSInternalContextBase, MDSGather>&, int)+0x1232) [0x557160d39352]
2018-10-06T23:06:23.137 INFO:tasks.ceph.mds.a.smithi023.stderr: 8: (MDLog::try_expire(LogSegment*, int)+0x66) [0x557160cd7636]
2018-10-06T23:06:23.137 INFO:tasks.ceph.mds.a.smithi023.stderr: 9: (MDLog::trim(int)+0x80c) [0x557160cd830c]
2018-10-06T23:06:23.137 INFO:tasks.ceph.mds.a.smithi023.stderr: 10: (MDCache::shutdown_pass()+0x82d) [0x557160b6960d]
2018-10-06T23:06:23.138 INFO:tasks.ceph.mds.a.smithi023.stderr: 11: (MDSRankDispatcher::tick()+0x2b8) [0x557160a4a068]
2018-10-06T23:06:23.138 INFO:tasks.ceph.mds.a.smithi023.stderr: 12: (FunctionContext::finish(int)+0x2c) [0x557160a348ec]
2018-10-06T23:06:23.138 INFO:tasks.ceph.mds.a.smithi023.stderr: 13: (Context::complete(int)+0x9) [0x557160a32789]
2018-10-06T23:06:23.138 INFO:tasks.ceph.mds.a.smithi023.stderr: 14: (SafeTimer::timer_thread()+0x189) [0x7f27414ceae9]
2018-10-06T23:06:23.138 INFO:tasks.ceph.mds.a.smithi023.stderr: 15: (SafeTimerThread::entry()+0xd) [0x7f27414d0ecd]
2018-10-06T23:06:23.138 INFO:tasks.ceph.mds.a.smithi023.stderr: 16: (()+0x7dd5) [0x7f273f0b8dd5]
2018-10-06T23:06:23.138 INFO:tasks.ceph.mds.a.smithi023.stderr: 17: (clone()+0x6d) [0x7f273dd6bb3d]
2018-10-06T23:06:23.138 INFO:tasks.ceph.mds.a.smithi023.stderr:*** Caught signal (Aborted) **
2018-10-06T23:06:23.138 INFO:tasks.ceph.mds.a.smithi023.stderr: in thread 7f2734577700 thread_name:safe_timer

From: /ceph/teuthology-archive/pdonnell-2018-10-06_01:15:54-multimds-wip-pdonnell-testing-20181005.225845-distro-basic-smithi/3107275/teuthology.log

Core: /ceph/teuthology-archive/pdonnell-2018-10-06_01:15:54-multimds-wip-pdonnell-testing-20181005.225845-distro-basic-smithi/3107275/remote/smithi023/coredump/1538867183.36265.core

Branch: https://github.com/batrick/ceph/tree/wip-pdonnell-testing-20181005.225845


Related issues 2 (0 open2 closed)

Copied to CephFS - Backport #37092: luminous: mds: "src/mds/MDLog.cc: 281: FAILED ceph_assert(!capped)" during max_mds thrashingResolvedWei-Chung ChengActions
Copied to CephFS - Backport #37093: mimic: mds: "src/mds/MDLog.cc: 281: FAILED ceph_assert(!capped)" during max_mds thrashingResolvedJonathan BrielmaierActions
Actions #2

Updated by Zheng Yan over 5 years ago

  • Status changed from New to Fix Under Review
Actions #3

Updated by Patrick Donnelly over 5 years ago

  • Status changed from Fix Under Review to Pending Backport
  • Assignee set to Zheng Yan
Actions #4

Updated by Nathan Cutler over 5 years ago

  • Copied to Backport #37092: luminous: mds: "src/mds/MDLog.cc: 281: FAILED ceph_assert(!capped)" during max_mds thrashing added
Actions #5

Updated by Nathan Cutler over 5 years ago

  • Copied to Backport #37093: mimic: mds: "src/mds/MDLog.cc: 281: FAILED ceph_assert(!capped)" during max_mds thrashing added
Actions #6

Updated by Patrick Donnelly over 5 years ago

  • Status changed from Pending Backport to Resolved
Actions

Also available in: Atom PDF