Bug #56802
crash: void MDLog::_submit_entry(LogEvent*, MDSLogContextBase*): assert(!mds->is_any_replay())
Status:
Duplicate
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Telemetry
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Labels (FS):
Pull request ID:
Crash signature (v1):
9b315c76704f48f58389355863edf69605a106605eaf0dae471bb8a33847c1da
aa1f46ee91981d0436cb9df53dc52e88e064034dfd359da45af492fbb73864d2
Crash signature (v2):
Description
Assert condition: !mds->is_any_replay()
Assert function: void MDLog::_submit_entry(LogEvent*, MDSLogContextBase*)
Sanitized backtrace:
MDLog::_submit_entry(LogEvent*, MDSLogContextBase*) Server::journal_close_session(Session*, int, Context*) Server::kill_session(Session*, Context*) Server::apply_blocklist() MDSRank::apply_blocklist(std::set<entity_addr_t, std::less<entity_addr_t>, std::allocator<entity_addr_t> > const&, unsigned int) MDSRankDispatcher::handle_osd_map() MDSDaemon::handle_core_message(boost::intrusive_ptr<Message const> const&) MDSDaemon::ms_dispatch2(boost::intrusive_ptr<Message> const&) DispatchQueue::entry() DispatchQueue::DispatchThread::entry()
Crash dump sample:
{ "assert_condition": "!mds->is_any_replay()", "assert_file": "mds/MDLog.cc", "assert_func": "void MDLog::_submit_entry(LogEvent*, MDSLogContextBase*)", "assert_line": 283, "assert_msg": "mds/MDLog.cc: In function 'void MDLog::_submit_entry(LogEvent*, MDSLogContextBase*)' thread 7fba8ec21700 time 2022-07-25T21:55:02.803985+0000\nmds/MDLog.cc: 283: FAILED ceph_assert(!mds->is_any_replay())", "assert_thread_name": "ms_dispatch", "backtrace": [ "/lib64/libpthread.so.0(+0x12ce0) [0x7fba96646ce0]", "gsignal()", "abort()", "(ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x7fba97666cd2]", "/usr/lib64/ceph/libceph-common.so.2(+0x283e95) [0x7fba97666e95]", "(MDLog::_submit_entry(LogEvent*, MDSLogContextBase*)+0x3f) [0x55aac823bfdf]", "(Server::journal_close_session(Session*, int, Context*)+0x78c) [0x55aac7f5a63c]", "(Server::kill_session(Session*, Context*)+0x212) [0x55aac7f5ad92]", "(Server::apply_blocklist()+0x10d) [0x55aac7f5b04d]", "(MDSRank::apply_blocklist(std::set<entity_addr_t, std::less<entity_addr_t>, std::allocator<entity_addr_t> > const&, unsigned int)+0x34) [0x55aac7f16d74]", "(MDSRankDispatcher::handle_osd_map()+0xf6) [0x55aac7f170b6]", "(MDSDaemon::handle_core_message(boost::intrusive_ptr<Message const> const&)+0x39b) [0x55aac7f0030b]", "(MDSDaemon::ms_dispatch2(boost::intrusive_ptr<Message> const&)+0xc3) [0x55aac7f00cc3]", "(DispatchQueue::entry()+0x14fa) [0x7fba978ed40a]", "(DispatchQueue::DispatchThread::entry()+0x11) [0x7fba979a4481]", "/lib64/libpthread.so.0(+0x81ca) [0x7fba9663c1ca]", "clone()" ], "ceph_version": "17.2.2", "crash_id": "2022-07-25T21:55:02.809868Z_e6390ae9-a7fa-4ec0-bf09-2698da00afe6", "entity_name": "mds.767cdeeef280795b3c477fe2cfad702b6dd095ad", "os_id": "centos", "os_name": "CentOS Stream", "os_version": "8", "os_version_id": "8", "process_name": "ceph-mds", "stack_sig": "aa1f46ee91981d0436cb9df53dc52e88e064034dfd359da45af492fbb73864d2", "timestamp": "2022-07-25T21:55:02.809868Z", "utsname_machine": "x86_64", "utsname_release": "3.10.0-1160.36.2.el7.x86_64", "utsname_sysname": "Linux", "utsname_version": "#1 SMP Wed Jul 21 11:57:15 UTC 2021" }
Related issues
History
#1 Updated by Telemetry Bot over 1 year ago
#2 Updated by Andreas Teuchert over 1 year ago
Maybe this is relevant information to reproduce the crash:
I have NFS Ganesha running to export CephFS and when I stop and start an NFS server (ceph orch daemon stop <daemon>
, then ceph orch daemon start <daemon>
) the standby-replay MDS (there's only a single active-standby pair at this time) crashes with the exact same crash as above (it seems to crash when the NFS server is started).
I'll be happy to provide any other information that could be helpful.
#3 Updated by Patrick Donnelly over 1 year ago
- Duplicates Bug #56012: mds: src/mds/MDLog.cc: 283: FAILED ceph_assert(!mds->is_ any_replay()) added
#4 Updated by Patrick Donnelly over 1 year ago
- Status changed from New to Duplicate
- Crash signature (v1) updated (diff)