Project

General

Profile

Actions

Bug #56850

closed

crash: void PaxosService::propose_pending(): assert(have_pending)

Added by Telemetry Bot over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Telemetry
Tags:
Backport:
Regression:
Yes
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):

e180965bf13ff8b359662a0570a7ddddcef1e81430600bfc86f58dc04ea73355


Description

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=106ff764dfe8a5f766a511a140a2931b405e14dd547b15bc4cfceced6c309638

Assert condition: have_pending
Assert function: void PaxosService::propose_pending()

Sanitized backtrace:

    PaxosService::propose_pending()
    KVMonitor::do_osd_new(uuid_d const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
    OSDMonitor::prepare_command_osd_new(boost::intrusive_ptr<MonOpRequest>, std::map<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, boost::variant<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > >, std::less<void>, std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> > const, boost::variant<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > > > > > const&, std::map<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >&, ceph::Formatter*)
    OSDMonitor::prepare_command_impl(boost::intrusive_ptr<MonOpRequest>, std::map<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, boost::variant<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > >, std::less<void>, std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> > const, boost::variant<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > > > > > const&)
    OSDMonitor::prepare_command(boost::intrusive_ptr<MonOpRequest>)
    OSDMonitor::prepare_update(boost::intrusive_ptr<MonOpRequest>)
    PaxosService::dispatch(boost::intrusive_ptr<MonOpRequest>)
    Monitor::handle_command(boost::intrusive_ptr<MonOpRequest>)
    Monitor::dispatch_op(boost::intrusive_ptr<MonOpRequest>)
    Monitor::_ms_dispatch(Message*)
    Monitor::handle_forward(boost::intrusive_ptr<MonOpRequest>)
    Monitor::dispatch_op(boost::intrusive_ptr<MonOpRequest>)
    Monitor::_ms_dispatch(Message*)
    Dispatcher::ms_dispatch2(boost::intrusive_ptr<Message> const&)
    DispatchQueue::entry()
    DispatchQueue::DispatchThread::entry()

Crash dump sample:
{
    "assert_condition": "have_pending",
    "assert_file": "mon/PaxosService.cc",
    "assert_func": "void PaxosService::propose_pending()",
    "assert_line": 193,
    "assert_msg": "mon/PaxosService.cc: In function 'void PaxosService::propose_pending()' thread 7f9b65ab5700 time 2022-07-08T00:45:34.399018+0000\nmon/PaxosService.cc: 193: FAILED ceph_assert(have_pending)",
    "assert_thread_name": "ms_dispatch",
    "backtrace": [
        "/lib64/libpthread.so.0(+0x12ce0) [0x7f9b6ff12ce0]",
        "gsignal()",
        "abort()",
        "(ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x7f9b71f80c32]",
        "/usr/lib64/ceph/libceph-common.so.2(+0x283df5) [0x7f9b71f80df5]",
        "(PaxosService::propose_pending()+0x50d) [0x55c699e5908d]",
        "(KVMonitor::do_osd_new(uuid_d const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x179) [0x55c699df9609]",
        "(OSDMonitor::prepare_command_osd_new(boost::intrusive_ptr<MonOpRequest>, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, boost::variant<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > >, std::less<void>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, boost::variant<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > > > > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >&, ceph::Formatter*)+0x1f2b) [0x55c699ea6dfb]",
        "(OSDMonitor::prepare_command_impl(boost::intrusive_ptr<MonOpRequest>, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, boost::variant<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > >, std::less<void>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, boost::variant<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > > > > > const&)+0x19f19) [0x55c699eca8a9]",
        "(OSDMonitor::prepare_command(boost::intrusive_ptr<MonOpRequest>)+0x10f) [0x55c699ed044f]",
        "(OSDMonitor::prepare_update(boost::intrusive_ptr<MonOpRequest>)+0x383) [0x55c699ee5983]",
        "(PaxosService::dispatch(boost::intrusive_ptr<MonOpRequest>)+0xa7d) [0x55c699e59c5d]",
        "(Monitor::handle_command(boost::intrusive_ptr<MonOpRequest>)+0x27ac) [0x55c699d1fecc]",
        "(Monitor::dispatch_op(boost::intrusive_ptr<MonOpRequest>)+0xb22) [0x55c699d24e32]",
        "(Monitor::_ms_dispatch(Message*)+0x457) [0x55c699d260c7]",
        "(Monitor::handle_forward(boost::intrusive_ptr<MonOpRequest>)+0x8e4) [0x55c699d276b4]",
        "(Monitor::dispatch_op(boost::intrusive_ptr<MonOpRequest>)+0x115d) [0x55c699d2546d]",
        "(Monitor::_ms_dispatch(Message*)+0x457) [0x55c699d260c7]",
        "(Dispatcher::ms_dispatch2(boost::intrusive_ptr<Message> const&)+0x5c) [0x55c699d5688c]",
        "(DispatchQueue::entry()+0x14fa) [0x7f9b7220743a]",
        "(DispatchQueue::DispatchThread::entry()+0x11) [0x7f9b722be581]",
        "/lib64/libpthread.so.0(+0x81ca) [0x7f9b6ff081ca]",
        "clone()" 
    ],
    "ceph_version": "17.2.1",
    "crash_id": "2022-07-08T00:45:34.402394Z_5b681d8d-a7aa-454f-87a8-71ff58dd0cc8",
    "entity_name": "mon.4fe0eeb4d1a92ea831e9e79b410d33b228f6cd3f",
    "os_id": "centos",
    "os_name": "CentOS Stream",
    "os_version": "8",
    "os_version_id": "8",
    "process_name": "ceph-mon",
    "stack_sig": "e180965bf13ff8b359662a0570a7ddddcef1e81430600bfc86f58dc04ea73355",
    "timestamp": "2022-07-08T00:45:34.402394Z",
    "utsname_machine": "x86_64",
    "utsname_release": "5.10.0-14-amd64",
    "utsname_sysname": "Linux",
    "utsname_version": "#1 SMP Debian 5.10.113-1 (2022-04-29)" 
}


Related issues 4 (1 open3 closed)

Related to RADOS - Bug #59813: crash: void PaxosService::propose_pending(): assert(have_pending)Pending BackportPatrick Donnelly

Actions
Has duplicate RADOS - Bug #56847: crash: void PaxosService::propose_pending(): assert(have_pending)Duplicate

Actions
Has duplicate RADOS - Bug #56848: crash: void PaxosService::propose_pending(): assert(have_pending)Duplicate

Actions
Has duplicate RADOS - Bug #56849: crash: void PaxosService::propose_pending(): assert(have_pending)Duplicate

Actions
Actions #1

Updated by Telemetry Bot over 1 year ago

  • Crash signature (v1) updated (diff)
  • Crash signature (v2) updated (diff)
  • Affected Versions v17.2.1 added
Actions #2

Updated by Kefu Chai over 1 year ago

  • Status changed from New to Fix Under Review
  • Assignee set to Kefu Chai
  • Regression changed from No to Yes
  • Pull request ID set to 47834
Actions #3

Updated by Kefu Chai over 1 year ago

  • Status changed from Fix Under Review to Resolved
Actions #4

Updated by Yaarit Hatuka 11 months ago

  • Has duplicate Bug #56847: crash: void PaxosService::propose_pending(): assert(have_pending) added
Actions #5

Updated by Yaarit Hatuka 11 months ago

  • Has duplicate Bug #56848: crash: void PaxosService::propose_pending(): assert(have_pending) added
Actions #6

Updated by Yaarit Hatuka 11 months ago

  • Has duplicate Bug #56849: crash: void PaxosService::propose_pending(): assert(have_pending) added
Actions #7

Updated by Telemetry Bot 11 months ago

  • Related to Bug #59813: crash: void PaxosService::propose_pending(): assert(have_pending) added
Actions

Also available in: Atom PDF