Project

General

Profile

Actions

Bug #56802

closed

crash: void MDLog::_submit_entry(LogEvent*, MDSLogContextBase*): assert(!mds->is_any_replay())

Added by Telemetry Bot almost 2 years ago. Updated almost 2 years ago.

Status:
Duplicate
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Telemetry
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Labels (FS):
Pull request ID:
Crash signature (v1):

9b315c76704f48f58389355863edf69605a106605eaf0dae471bb8a33847c1da
aa1f46ee91981d0436cb9df53dc52e88e064034dfd359da45af492fbb73864d2


Description

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=902003e195a320e2927d5e3990bff9baebe6292ab8d7abd1e44b9fc2ed7d2f9f

Assert condition: !mds->is_any_replay()
Assert function: void MDLog::_submit_entry(LogEvent*, MDSLogContextBase*)

Sanitized backtrace:

    MDLog::_submit_entry(LogEvent*, MDSLogContextBase*)
    Server::journal_close_session(Session*, int, Context*)
    Server::kill_session(Session*, Context*)
    Server::apply_blocklist()
    MDSRank::apply_blocklist(std::set<entity_addr_t, std::less<entity_addr_t>, std::allocator<entity_addr_t> > const&, unsigned int)
    MDSRankDispatcher::handle_osd_map()
    MDSDaemon::handle_core_message(boost::intrusive_ptr<Message const> const&)
    MDSDaemon::ms_dispatch2(boost::intrusive_ptr<Message> const&)
    DispatchQueue::entry()
    DispatchQueue::DispatchThread::entry()

Crash dump sample:
{
    "assert_condition": "!mds->is_any_replay()",
    "assert_file": "mds/MDLog.cc",
    "assert_func": "void MDLog::_submit_entry(LogEvent*, MDSLogContextBase*)",
    "assert_line": 283,
    "assert_msg": "mds/MDLog.cc: In function 'void MDLog::_submit_entry(LogEvent*, MDSLogContextBase*)' thread 7fba8ec21700 time 2022-07-25T21:55:02.803985+0000\nmds/MDLog.cc: 283: FAILED ceph_assert(!mds->is_any_replay())",
    "assert_thread_name": "ms_dispatch",
    "backtrace": [
        "/lib64/libpthread.so.0(+0x12ce0) [0x7fba96646ce0]",
        "gsignal()",
        "abort()",
        "(ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x7fba97666cd2]",
        "/usr/lib64/ceph/libceph-common.so.2(+0x283e95) [0x7fba97666e95]",
        "(MDLog::_submit_entry(LogEvent*, MDSLogContextBase*)+0x3f) [0x55aac823bfdf]",
        "(Server::journal_close_session(Session*, int, Context*)+0x78c) [0x55aac7f5a63c]",
        "(Server::kill_session(Session*, Context*)+0x212) [0x55aac7f5ad92]",
        "(Server::apply_blocklist()+0x10d) [0x55aac7f5b04d]",
        "(MDSRank::apply_blocklist(std::set<entity_addr_t, std::less<entity_addr_t>, std::allocator<entity_addr_t> > const&, unsigned int)+0x34) [0x55aac7f16d74]",
        "(MDSRankDispatcher::handle_osd_map()+0xf6) [0x55aac7f170b6]",
        "(MDSDaemon::handle_core_message(boost::intrusive_ptr<Message const> const&)+0x39b) [0x55aac7f0030b]",
        "(MDSDaemon::ms_dispatch2(boost::intrusive_ptr<Message> const&)+0xc3) [0x55aac7f00cc3]",
        "(DispatchQueue::entry()+0x14fa) [0x7fba978ed40a]",
        "(DispatchQueue::DispatchThread::entry()+0x11) [0x7fba979a4481]",
        "/lib64/libpthread.so.0(+0x81ca) [0x7fba9663c1ca]",
        "clone()" 
    ],
    "ceph_version": "17.2.2",
    "crash_id": "2022-07-25T21:55:02.809868Z_e6390ae9-a7fa-4ec0-bf09-2698da00afe6",
    "entity_name": "mds.767cdeeef280795b3c477fe2cfad702b6dd095ad",
    "os_id": "centos",
    "os_name": "CentOS Stream",
    "os_version": "8",
    "os_version_id": "8",
    "process_name": "ceph-mds",
    "stack_sig": "aa1f46ee91981d0436cb9df53dc52e88e064034dfd359da45af492fbb73864d2",
    "timestamp": "2022-07-25T21:55:02.809868Z",
    "utsname_machine": "x86_64",
    "utsname_release": "3.10.0-1160.36.2.el7.x86_64",
    "utsname_sysname": "Linux",
    "utsname_version": "#1 SMP Wed Jul 21 11:57:15 UTC 2021" 
}


Related issues 1 (0 open1 closed)

Is duplicate of CephFS - Bug #56012: mds: src/mds/MDLog.cc: 283: FAILED ceph_assert(!mds->is_ any_replay())ResolvedKotresh Hiremath Ravishankar

Actions
Actions #1

Updated by Telemetry Bot almost 2 years ago

  • Crash signature (v1) updated (diff)
  • Crash signature (v2) updated (diff)
  • Affected Versions v17.2.1, v17.2.2 added
Actions #2

Updated by Andreas Teuchert almost 2 years ago

Maybe this is relevant information to reproduce the crash:

I have NFS Ganesha running to export CephFS and when I stop and start an NFS server (ceph orch daemon stop <daemon>, then ceph orch daemon start <daemon>) the standby-replay MDS (there's only a single active-standby pair at this time) crashes with the exact same crash as above (it seems to crash when the NFS server is started).

I'll be happy to provide any other information that could be helpful.

Actions #3

Updated by Patrick Donnelly almost 2 years ago

  • Is duplicate of Bug #56012: mds: src/mds/MDLog.cc: 283: FAILED ceph_assert(!mds->is_ any_replay()) added
Actions #4

Updated by Patrick Donnelly almost 2 years ago

  • Status changed from New to Duplicate
  • Crash signature (v1) updated (diff)
Actions

Also available in: Atom PDF