Project

General

Profile

Bug #56802

crash: void MDLog::_submit_entry(LogEvent*, MDSLogContextBase*): assert(!mds->is_any_replay())

Added by Telemetry Bot over 1 year ago. Updated over 1 year ago.

Status:
Duplicate
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Telemetry
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Labels (FS):
Pull request ID:
Crash signature (v1):

9b315c76704f48f58389355863edf69605a106605eaf0dae471bb8a33847c1da
aa1f46ee91981d0436cb9df53dc52e88e064034dfd359da45af492fbb73864d2


Description

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=902003e195a320e2927d5e3990bff9baebe6292ab8d7abd1e44b9fc2ed7d2f9f

Assert condition: !mds->is_any_replay()
Assert function: void MDLog::_submit_entry(LogEvent*, MDSLogContextBase*)

Sanitized backtrace:

    MDLog::_submit_entry(LogEvent*, MDSLogContextBase*)
    Server::journal_close_session(Session*, int, Context*)
    Server::kill_session(Session*, Context*)
    Server::apply_blocklist()
    MDSRank::apply_blocklist(std::set<entity_addr_t, std::less<entity_addr_t>, std::allocator<entity_addr_t> > const&, unsigned int)
    MDSRankDispatcher::handle_osd_map()
    MDSDaemon::handle_core_message(boost::intrusive_ptr<Message const> const&)
    MDSDaemon::ms_dispatch2(boost::intrusive_ptr<Message> const&)
    DispatchQueue::entry()
    DispatchQueue::DispatchThread::entry()

Crash dump sample:
{
    "assert_condition": "!mds->is_any_replay()",
    "assert_file": "mds/MDLog.cc",
    "assert_func": "void MDLog::_submit_entry(LogEvent*, MDSLogContextBase*)",
    "assert_line": 283,
    "assert_msg": "mds/MDLog.cc: In function 'void MDLog::_submit_entry(LogEvent*, MDSLogContextBase*)' thread 7fba8ec21700 time 2022-07-25T21:55:02.803985+0000\nmds/MDLog.cc: 283: FAILED ceph_assert(!mds->is_any_replay())",
    "assert_thread_name": "ms_dispatch",
    "backtrace": [
        "/lib64/libpthread.so.0(+0x12ce0) [0x7fba96646ce0]",
        "gsignal()",
        "abort()",
        "(ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x7fba97666cd2]",
        "/usr/lib64/ceph/libceph-common.so.2(+0x283e95) [0x7fba97666e95]",
        "(MDLog::_submit_entry(LogEvent*, MDSLogContextBase*)+0x3f) [0x55aac823bfdf]",
        "(Server::journal_close_session(Session*, int, Context*)+0x78c) [0x55aac7f5a63c]",
        "(Server::kill_session(Session*, Context*)+0x212) [0x55aac7f5ad92]",
        "(Server::apply_blocklist()+0x10d) [0x55aac7f5b04d]",
        "(MDSRank::apply_blocklist(std::set<entity_addr_t, std::less<entity_addr_t>, std::allocator<entity_addr_t> > const&, unsigned int)+0x34) [0x55aac7f16d74]",
        "(MDSRankDispatcher::handle_osd_map()+0xf6) [0x55aac7f170b6]",
        "(MDSDaemon::handle_core_message(boost::intrusive_ptr<Message const> const&)+0x39b) [0x55aac7f0030b]",
        "(MDSDaemon::ms_dispatch2(boost::intrusive_ptr<Message> const&)+0xc3) [0x55aac7f00cc3]",
        "(DispatchQueue::entry()+0x14fa) [0x7fba978ed40a]",
        "(DispatchQueue::DispatchThread::entry()+0x11) [0x7fba979a4481]",
        "/lib64/libpthread.so.0(+0x81ca) [0x7fba9663c1ca]",
        "clone()" 
    ],
    "ceph_version": "17.2.2",
    "crash_id": "2022-07-25T21:55:02.809868Z_e6390ae9-a7fa-4ec0-bf09-2698da00afe6",
    "entity_name": "mds.767cdeeef280795b3c477fe2cfad702b6dd095ad",
    "os_id": "centos",
    "os_name": "CentOS Stream",
    "os_version": "8",
    "os_version_id": "8",
    "process_name": "ceph-mds",
    "stack_sig": "aa1f46ee91981d0436cb9df53dc52e88e064034dfd359da45af492fbb73864d2",
    "timestamp": "2022-07-25T21:55:02.809868Z",
    "utsname_machine": "x86_64",
    "utsname_release": "3.10.0-1160.36.2.el7.x86_64",
    "utsname_sysname": "Linux",
    "utsname_version": "#1 SMP Wed Jul 21 11:57:15 UTC 2021" 
}


Related issues

Duplicates CephFS - Bug #56012: mds: src/mds/MDLog.cc: 283: FAILED ceph_assert(!mds->is_ any_replay()) Resolved

History

#1 Updated by Telemetry Bot over 1 year ago

  • Crash signature (v1) updated (diff)
  • Crash signature (v2) updated (diff)
  • Affected Versions v17.2.1, v17.2.2 added

#2 Updated by Andreas Teuchert over 1 year ago

Maybe this is relevant information to reproduce the crash:

I have NFS Ganesha running to export CephFS and when I stop and start an NFS server (ceph orch daemon stop <daemon>, then ceph orch daemon start <daemon>) the standby-replay MDS (there's only a single active-standby pair at this time) crashes with the exact same crash as above (it seems to crash when the NFS server is started).

I'll be happy to provide any other information that could be helpful.

#3 Updated by Patrick Donnelly over 1 year ago

  • Duplicates Bug #56012: mds: src/mds/MDLog.cc: 283: FAILED ceph_assert(!mds->is_ any_replay()) added

#4 Updated by Patrick Donnelly over 1 year ago

  • Status changed from New to Duplicate
  • Crash signature (v1) updated (diff)

Also available in: Atom PDF