Project

General

Profile

Actions

Bug #55773

closed

Assertion failure (ceph_assert(have_pending)) when creating new OSDs during OSD deployment.

Added by Sridhar Seshasayee almost 2 years ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Category:
Monitor
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
quincy, pacific
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Monitor
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

See https://bugzilla.redhat.com/show_bug.cgi?id=2086419 for more details.

Assertion Failure

2022-05-13T13:14:04.103+0000 7f1cccf9a700 -1 /builddir/build/BUILD/ceph-16.2.7/src/mon/PaxosService.cc: In function 'void PaxosService::propose_pending()' thread 7f1cccf9a700 time 2022-05-13T13:14:04.101637+0000
/builddir/build/BUILD/ceph-16.2.7/src/mon/PaxosService.cc: 193: FAILED ceph_assert(have_pending)

Backtrace

>>> ceph version 16.2.7-112.el8cp (e18db2ff03ac60c64a18f3315c032b9d5a0a3b8f) pacific (stable)
>>> 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f1cdc0dccfe]
>>> 2: /usr/lib64/ceph/libceph-common.so.2(+0x276f18) [0x7f1cdc0dcf18]
>>> 3: (PaxosService::propose_pending()+0x50d) [0x561fa6f61add]
>>> 4: (KVMonitor::do_osd_new(uuid_d const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x1ba) [0x561fa6f0514a]
>>> 5: (OSDMonitor::prepare_command_osd_new(boost::intrusive_ptr<MonOpRequest>, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, boost::variant<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > >, std::less<void>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, boost::variant<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > > > > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >&, ceph::Formatter*)+0x1fdb) [0x561fa6faec8b]
>>> 6: (OSDMonitor::prepare_command_impl(boost::intrusive_ptr<MonOpRequest>, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, boost::variant<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > >, std::less<void>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, boost::variant<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, double, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<long, std::allocator<long> >, std::vector<double, std::allocator<double> > > > > > const&)+0x17278) [0x561fa6fd0f68]
>>> 7: (OSDMonitor::prepare_command(boost::intrusive_ptr<MonOpRequest>)+0xf4) [0x561fa6fdd664]
>>> 8: (OSDMonitor::prepare_update(boost::intrusive_ptr<MonOpRequest>)+0x373) [0x561fa6fe1943]
>>> 9: (PaxosService::dispatch(boost::intrusive_ptr<MonOpRequest>)+0xa6d) [0x561fa6f6269d]
>>> 10: (PaxosService::C_RetryMessage::_finish(int)+0x64) [0x561fa6eb5514]
>>> 11: (C_MonOp::finish(int)+0x49) [0x561fa6e52829]
>>> 12: (Context::complete(int)+0xd) [0x561fa6e4fcfd]
>>> 13: (void finish_contexts<std::__cxx11::list<Context*, std::allocator<Context*> > >(ceph::common::CephContext*, std::__cxx11::list<Context*, std::allocator<Context*> >&, int)+0xa5) [0x561fa6e7c1f5]
>>> 14: (PaxosService::_active()+0x1cb) [0x561fa6f635ab]
>>> 15: (Context::complete(int)+0xd) [0x561fa6e4fcfd]
>>> 16: (void finish_contexts<std::__cxx11::list<Context*, std::allocator<Context*> > >(ceph::common::CephContext*, std::__cxx11::list<Context*, std::allocator<Context*> >&, int)+0xa5) [0x561fa6e7c1f5]
>>> 17: (Paxos::finish_round()+0x169) [0x561fa6f59ea9]
>>> 18: (Paxos::commit_finish()+0x8c0) [0x561fa6f5c9d0]
>>> 19: (C_Committed::finish(int)+0x45) [0x561fa6f605f5]
>>> 20: (Context::complete(int)+0xd) [0x561fa6e4fcfd]
>>> 21: (MonitorDBStore::C_DoTransaction::finish(int)+0x98) [0x561fa6f60318]
>>> 22: (Context::complete(int)+0xd) [0x561fa6e4fcfd]
>>> 23: (Finisher::finisher_thread_entry()+0x1a5) [0x7f1cdc17e2a5]
>>> 24: /lib64/libpthread.so.0(+0x81cf) [0x7f1cd9e111cf]
>>> 25: clone()

Related issues 2 (0 open2 closed)

Copied to RADOS - Backport #56059: pacific: Assertion failure (ceph_assert(have_pending)) when creating new OSDs during OSD deployment.ResolvedSridhar SeshasayeeActions
Copied to RADOS - Backport #56060: quincy: Assertion failure (ceph_assert(have_pending)) when creating new OSDs during OSD deployment.ResolvedSridhar SeshasayeeActions
Actions

Also available in: Atom PDF