Project

General

Profile

Bug #43073

ceph_perf_msgr_client crashed

Added by yong xing 3 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
AsyncMessenger
Target version:
% Done:

0%

Source:
Community (user)
Tags:
Messengers
Backport:
v14.2.4
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

ceph_perf_msgr_client crashs as follows,

#0 _GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00007fffec59f801 in __GI_abort () at abort.c:79
#2 0x00007fffee6f1757 in ceph::
_ceph_assert_fail (assertion=0x7fffef04d50b "messenger->auth_client", file=0x7fffef04c6b0 "/root/code/ceph/src/msg/async/ProtocolV2.cc", line=1705,
func=0x7fffef051680 <ProtocolV2::send_auth_request(std::vector<unsigned int, std::allocator<unsigned int> >&)::__PRETTY_FUNCTION__> "Ct<ProtocolV2>* ProtocolV2::send_auth_request(std::vector<unsigned int>&)") at /root/code/ceph/src/common/assert.cc:73
#3 0x00007fffee6f180a in ceph::__ceph_assert_fail (ctx=...) at /root/code/ceph/src/common/assert.cc:78
#4 0x00007fffeea89137 in ProtocolV2::send_auth_request (this=0x555556cbf080, allowed_methods=std::vector of length 0, capacity 0) at /root/code/ceph/src/msg/async/ProtocolV2.cc:1705
#5 0x00007fffeea9b029 in ProtocolV2::send_auth_request (this=0x555556cbf080) at /root/code/ceph/src/msg/async/ProtocolV2.h:215
#6 0x00007fffeea88ede in ProtocolV2::post_client_banner_exchange (this=0x555556cbf080) at /root/code/ceph/src/msg/async/ProtocolV2.cc:1699
#7 0x00007fffeeab0e64 in CtFun<ProtocolV2>::_call<>(ProtocolV2*, std::integer_sequence<unsigned long>) const (this=0x555556cbf540, foo=0x555556cbf080)
at /root/code/ceph/src/msg/async/Protocol.h:37
#8 0x00007fffeeab0df6 in CtFun<ProtocolV2>::call (this=0x555556cbf540, foo=0x555556cbf080) at /root/code/ceph/src/msg/async/Protocol.h:45
#9 0x00007fffeea76c1c in ProtocolV2::run_continuation (this=0x555556cbf080, continuation=...) at /root/code/ceph/src/msg/async/ProtocolV2.cc:45
#10 0x00007fffeea7dbaa in ProtocolV2::<lambda(char*, int)>::operator()(char , int) const (_closure=0x555556026c98, buffer=0x555556032140 "ceph v2\n\020", r=0)
at /root/code/ceph/src/msg/async/ProtocolV2.cc:719
#11 0x00007fffeea98b54 in std::_Function_handler<void(char
, long int), ProtocolV2::read(CONTINUATION_RXBPTR_TYPE<ProtocolV2>&, rx_buffer_t&&)::<lambda(char*, int)> >::_M_invoke(const std::_Any_data &, char &&, long &&) (_functor=..., _args#0=@0x7fffe73b3590: 0x555556032140 "ceph v2\n\020", __args#1=@0x7fffe73b3588: 0) at /usr/include/c++/7/bits/std_function.h:316
#12 0x00007fffeea3afbd in std::function<void (char
, long)>::operator()(char*, long) const (this=0x555556026c98, __args#0=0x555556032140 "ceph v2\n\020", __args#1=0)
at /usr/include/c++/7/bits/std_function.h:706
#13 0x00007fffeea36119 in AsyncConnection::process (this=0x555556026880) at /root/code/ceph/src/msg/async/AsyncConnection.cc:450
#14 0x00007fffeea3a1b2 in C_handle_read::do_request (this=0x55555600c4e0, fd_or_id=12) at /root/code/ceph/src/msg/async/AsyncConnection.cc:71
#15 0x00007fffeeab464e in EventCenter::process_events (this=0x55555600ebc0, timeout_microseconds=10000000, working_dur=0x7fffe73b3948) at /root/code/ceph/src/msg/async/Event.cc:415
#16 0x00007fffeeac1d88 in NetworkStack::<lambda()>::operator()(void) const (
_closure=0x5555560c0458) at /root/code/ceph/src/msg/async/Stack.cc:53
#17 0x00007fffeeac325b in std::_Function_handler<void(), NetworkStack::add_thread(unsigned int)::<lambda()> >::_M_invoke(const std::_Any_data &) (_functor=...)
at /usr/include/c++/7/bits/std_function.h:316
#18 0x00007fffeeac05de in std::function<void ()>::operator()() const (this=0x5555560c0458) at /usr/include/c++/7/bits/std_function.h:706
#19 0x00007fffeeabfe7b in std::
_invoke_impl<void, std::function<void ()>>(std::__invoke_other, std::function<void ()>&&) (_f=...) at /usr/include/c++/7/bits/invoke.h:60
#20 0x00007fffeeabf608 in std::
_invoke<std::function<void ()>>(std::function<void ()>&&) (__fn=...) at /usr/include/c++/7/bits/invoke.h:95
#21 0x00007fffeeac153e in std::thread::_Invoker<std::tuple<std::function<void ()> > >::_M_invoke<0ul>(std::_Index_tuple<0ul>) (this=0x5555560c0458) at /usr/include/c++/7/thread:234
#22 0x00007fffeeac150f in std::thread::_Invoker<std::tuple<std::function<void ()> > >::operator()() (this=0x5555560c0458) at /usr/include/c++/7/thread:243
#23 0x00007fffeeac14ee in std::thread::_State_impl<std::thread::_Invoker<std::tuple<std::function<void ()> > > >::_M_run() (this=0x5555560c0450) at /usr/include/c++/7/thread:186
#24 0x00007fffecc2557f in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#25 0x00007ffff7bbd6db in start_thread (arg=0x7fffe73b5700) at pthread_create.c:463
#26 0x00007fffec68088f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

(gdb) f 4
#4 0x00007fffeea89137 in ProtocolV2::send_auth_request (this=0x555556cbf080, allowed_methods=std::vector of length 0, capacity 0) at /root/code/ceph/src/msg/async/ProtocolV2.cc:1705
1705 ceph_assert(messenger->auth_client);
(gdb) p messenger
$2 = (AsyncMessenger *) 0x555556cce000
(gdb) p messenger.auth_client
$3 = (AuthClient *) 0x0

ceph/src/msg/async/ProtocolV2.cc

1702 CtPtr ProtocolV2::send_auth_request(std::vector<uint32_t> &allowed_methods) {
1703 ldout(cct, 20) << func << " peer_type " << (int)connection->peer_type
1704 << " auth_client " << messenger->auth_client << dendl;
1705 ceph_assert(messenger->auth_client);

Also available in: Atom PDF