Project

General

Profile

Actions

Bug #42805

closed

SIGSEGV error occurs in checksummer::sum

Added by chunsong feng over 4 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
AsyncMessenger
Target version:
% Done:

0%

Source:
Community (dev)
Tags:
Backport:
Regression:
No
Severity:
1 - critical
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

The message V2 consists of preamble, payload and epilogue. The payload length may be 0. When the checksum is calculated for a zero lenght fragment, a SIGSEGV error occurs.
Thread 5 "msgr-worker-0" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xffffbd3dd9f0 (LWP 744384)]
checksummer::sum (this=this@entry=0xffffbd3dc910, data=<optimized out>, len=18446744073709551615)
at ./src/msg/async/dpdk/IPChecksum.cc:35
35 csum = ntohq(p64+);
(gdb) bt
#0 checksummer::sum (this=this@entry=0xffffbd3dc910, data=<optimized out>, len=18446744073709551615)
at ./src/msg/async/dpdk/IPChecksum.cc:35
#1 0x0000aaaaac356b50 in checksummer::sum (this=this@entry=0xffffbd3dc910, p=...)
at ./src/msg/async/dpdk/IPChecksum.cc:62
#2 0x0000aaaaac35b984 in tcp<ipv4_traits>::tcb::output_one (this=this@entry=0xaaaaba24e340,
data_retransmit=data_retransmit@entry=false) at /usr/include/c++/9/bits/unique_ptr.h:357
#3 0x0000aaaaac363398 in tcp<ipv4_traits>::tcb::get_packet (this=0xaaaaba24e340) at ./src/common/Tub.h:115
#4 tcp<ipv4_traits>::tcp(CephContext
, ipv4_l4<(ip_protocol_num)6>&, EventCenter*)::{lambda()#1}::operator()() (
this=0xaaaab5621800) at ./src/msg/async/dpdk/TCP.h:785
#5 0x0000aaaaac363510 in std::_Function_handler<Tub<ipv4_traits::l4packet> (), tcp<ipv4_traits>::tcp(CephContext*, ipv4_l4<(ip_protocol_num)6>&, EventCenter*)::{lambda()#1}>::_M_invoke(std::_Any_data const&) (functor=...)
at /usr/include/c++/9/bits/std_function.h:151
#6 0x0000aaaaac359e0c in std::function<Tub<ipv4_traits::l4packet> ()>::operator()() const (this=<optimized out>)
at /usr/include/c++/9/bits/std_function.h:685
#7 ipv4_l4<(ip_protocol_num)6>::register_packet_provider(std::function<Tub<ipv4_traits::l4packet> ()>)::{lambda()#1}::operator()() const (this=<optimized out>) at ./src/msg/async/dpdk/IP.h:348
#8 std::_Function_handler<Tub<ipv4_traits::l4packet> (), ipv4_l4<(ip_protocol_num)6>::register_packet_provider(std::function<Tub<ipv4_traits::l4packet> ()>)::{lambda()#1}>::_M_invoke(std::_Any_data const&) (_functor=...)
at /usr/include/c++/9/bits/std_function.h:286
#9 0x0000aaaaac34d2e8 in std::function<Tub<ipv4_traits::l4packet> ()>::operator()() const (this=<optimized out>)
at /usr/include/c++/9/bits/std_function.h:685
#10 ipv4::get_packet (this=0xaaaab67d00b8) at ./src/msg/async/dpdk/IP.cc:333
#11 0x0000aaaaac34d694 in ipv4::<lambda()>::operator() (
_closure=<optimized out>) at ./src/msg/async/dpdk/IP.cc:80
#12 std::_Function_handler<Tub<l3_protocol::l3packet>(), ipv4::ipv4(CephContext*, EventCenter*, interface*)::<lambda()> >::_M_invoke(const std::_Any_data &) (_functor=...) at /usr/include/c++/9/bits/std_function.h:286
#13 0x0000aaaaac354328 in std::function<Tub<l3_protocol::l3packet> ()>::operator()() const (this=<optimized out>)
at /usr/include/c++/9/bits/std_function.h:685
#14 interface::<lambda()>::operator() (
_closure=0xaaaab621b6a0, _closure=0xaaaab621b6a0)
at ./src/msg/async/dpdk/net.cc:57
#15 std::_Function_handler<Tub<Packet>(), interface::interface(CephContext*, std::shared_ptr<DPDKDevice>, EventCenter*)::<lambda()> >::_M_invoke(const std::_Any_data &) (
_functor=...) at /usr/include/c++/9/bits/std_function.h:286
#16 0x0000aaaaac36a6c4 in std::function<Tub<Packet> ()>::operator()() const (this=0xaaaab621b6a0)
at /usr/include/c++/9/bits/std_function.h:685
#17 DPDKQueuePair::poll_tx (this=0xaaaab5608f00) at ./src/msg/async/dpdk/DPDK.cc:657
#18 0x0000aaaaac3469f0 in DPDKQueuePair::DPDKTXPoller::poll (this=<optimized out>) at ./src/msg/async/dpdk/DPDK.h:675
#19 0x0000aaaaabcc65d0 in EventCenter::process_events (this=this@entry=0xaaaab5579048,
timeout_microseconds=<optimized out>, working_dur=working_dur@entry=0xffffbd3dd010)
at /usr/include/c++/9/bits/stl_vector.h:1040
---Type <return> to continue, or q <return> to quit---
#20 0x0000aaaaabccc380 in NetworkStack::<lambda()>::operator() (_closure=0xaaaab55c45a0, __closure=0xaaaab55c45a0)
at ./src/msg/async/Stack.cc:53
#21 std::_Function_handler<void(), NetworkStack::add_thread(unsigned int)::<lambda()> >::_M_invoke(const std::_Any_data &)
(
_functor=...) at /usr/include/c++/9/bits/std_function.h:300
#22 0x0000aaaaac342d20 in std::function<void ()>::operator()() const (this=<optimized out>)
at /usr/include/c++/9/bits/std_function.h:685
#23 dpdk_thread_adaptor (f=<optimized out>) at ./src/msg/async/dpdk/DPDKStack.cc:60
#24 0x0000aaaaac3847b8 in eal_thread_loop ()
#25 0x0000ffffbf214088 in start_thread (arg=0xffffbe3deb0f) at pthread_create.c:463
#26 0x0000ffffbee324ec in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78
(gdb) bt full
#0 checksummer::sum (this=this@entry=0xffffbd3dc910, data=<optimized out>, len=18446744073709551615)
at ./src/msg/async/dpdk/IPChecksum.cc:35
orig_len = 0
p64 = 0xaaaaba466005
p16 = <optimized out>
p8 = <optimized out>
#1 0x0000aaaaac356b50 in checksummer::sum (this=this@entry=0xffffbd3dc910, p=...)
at ./src/msg/async/dpdk/IPChecksum.cc:62
f = @0xaaaab9bf2e60: {base = 0xaaaab9b5625c "\002", size = 0}
__for_range = <optimized out>
__for_begin = 0xaaaab9bf2e60
__for_end = 0xaaaab9bf2eb0
#2 0x0000aaaaac35b984 in tcp<ipv4_traits>::tcb::output_one (this=this@entry=0xaaaaba24e340,
data_retransmit=data_retransmit@entry=false) at /usr/include/c++/9/bits/unique_ptr.h:357
p = {static internal_data_size = 112, static default_nr_frags = 4, _impl = std::unique_ptr<Packet::impl> = {
get() = 0xaaaab9bf2d80}}
clone = {static internal_data_size = 112, static default_nr_frags = 4, _impl = std::unique_ptr<Packet::impl> = {
get() = 0xaaaab566c400}}
len = 498
syn_on = <optimized out>
ack_on = <optimized out>
options_size = <optimized out>
th = 0xaaaab9bf2e04
seq = <optimized out>
fin_on = false
oi = {protocol = 1, needs_csum = false, ip_hdr_len = 20 '\024', tcp_hdr_len = 20 '\024', udp_hdr_len = 8 '\b',
needs_ip_csum = false, reassembled = false, tso_seg_size = 0, vlan_tci = {object = 0xa, raw = {<optimized out>,
<optimized out>}, occupied = false}}
csum = {csum = 2301032513407186491723761, odd = true}
pseudo_hdr_seg_len = <optimized out>
#3 0x0000aaaaac363398 in tcp<ipv4_traits>::tcb::get_packet (this=0xaaaaba24e340) at ./src/common/Tub.h:115
p = <optimized out>
__PRETTY_FUNCTION
= <optimized out>
p = <optimized out>
assert_data_ctx = {assertion = 0xaaaaac615250 "!_packetq.empty()",
file = 0xaaaaac615698 "/tmp/release/Ubuntu/WORKDIR/ceph-15.0.0-4703-gb954fa104c/src/msg/async/dpdk/TCP.h",
line = 1463,
function = 0xaaaaac615268 "Tub<typename InetTraits::l4packet> tcp<InetTraits>::tcb::get_packet() [with InetTraits---Type <return> to continue, or q <return> to quit-- = ipv4_traits; typename InetTraits::l4packet = ipv4_traits::l4packet]"}
#4 tcp<ipv4_traits>::tcp(CephContext*, ipv4_l4<(ip_protocol_num)6>&, EventCenter*)::{lambda()#1}::operator()() (
this=0xaaaab5621800) at ./src/msg/async/dpdk/TCP.h:785
tcb = <optimized out>
dst = {mac = {_M_elems = "\276\273\243N\a~"}}
l4p = {object = 0x0, raw = {<optimized out> <repeats 24 times>}, occupied = false}
c = 1
tcb_polled = 2722
this = 0xaaaab5621800
#5 0x0000aaaaac363510 in std::_Function_handler<Tub<ipv4_traits::l4packet> (), tcp<ipv4_traits>::tcp(CephContext*, ipv4_l4<(ip_protocol_num)6>&, EventCenter*)::{lambda()#1}>::_M_invoke(std::_Any_data const&) (_functor=...)
at /usr/include/c++/9/bits/std_function.h:151
No locals.
#6 0x0000aaaaac359e0c in std::function<Tub<ipv4_traits::l4packet> ()>::operator()() const (this=<optimized out>)
at /usr/include/c++/9/bits/std_function.h:685
No locals.
#7 ipv4_l4<(ip_protocol_num)6>::register_packet_provider(std::function<Tub<ipv4_traits::l4packet> ()>)::{lambda()#1}::operator()() const (this=<optimized out>) at ./src/msg/async/dpdk/IP.h:348
l4p = {object = 0xffffbd3dca78,
raw = "\214\343\v\277\377\377\000\000\000hh\265\252\252\000\000hxh\265\252\252\000", occupied = false}
func = <optimized out>
func = <optimized out>
l4p = <optimized out>
#8 std::_Function_handler<Tub<ipv4_traits::l4packet> (), ipv4_l4<(ip_protocol_num)6>::register_packet_provider(std::function<Tub<ipv4_traits::l4packet> ()>)::{lambda()#1}>::_M_invoke(std::_Any_data const&) (
_functor=...)
at /usr/include/c++/9/bits/std_function.h:286
No locals.
#9 0x0000aaaaac34d2e8 in std::function<Tub<ipv4_traits::l4packet> ()>::operator()() const (this=<optimized out>)
at /usr/include/c++/9/bits/std_function.h:685
No locals.
#10 ipv4::get_packet (this=0xaaaab67d00b8) at ./src/msg/async/dpdk/IP.cc:333
l4p = {object = 0xffffbd3dcb28,
raw = "\224m6\254\252\252\000\000\250\347o\277\377\377\000\000`\314=\275\377\377\000", occupied = false}
i = 1
p = {object = 0xffffbd3dcc60, raw = "\000\b\000\030-\000\000u\000\000\000\000\000\000\000", occupied = false}
#11 0x0000aaaaac34d694 in ipv4::<lambda()>::operator() (_closure=<optimized out>) at ./src/msg/async/dpdk/IP.cc:80
this = <optimized out>
this = <optimized out>
---Type <return> to continue, or q <return> to quit---
#12 std::_Function_handler<Tub<l3_protocol::l3packet>(), ipv4::ipv4(CephContext*, EventCenter*, interface*)::<lambda()> >::_M_invoke(const std::_Any_data &) (
_functor=...) at /usr/include/c++/9/bits/std_function.h:286
No locals.
#13 0x0000aaaaac354328 in std::function<Tub<l3_protocol::l3packet> ()>::operator()() const (this=<optimized out>)
at /usr/include/c++/9/bits/std_function.h:685
No locals.
#14 interface::<lambda()>::operator() (_closure=0xaaaab621b6a0, __closure=0xaaaab621b6a0)
at ./src/msg/async/dpdk/net.cc:57
l3p = {object = 0xffffbd3dcc60, raw = "\000\b\000\030-\000\000u\000\000\000\000\000\000\000", occupied = false}
i = 1
p = <optimized out>
qid = <optimized out>
idx = <optimized out>
this = <optimized out>
qid = <optimized out>
idx = <optimized out>
this = <optimized out>
p = <optimized out>
i = <optimized out>
l3p = <optimized out>
l3pv = <optimized out>
eh = <optimized out>
should_gather = <optimized out>
_dout_e = <optimized out>
_dout_cct = <optimized out>
_dout = <optimized out>
#15 std::_Function_handler<Tub<Packet>(), interface::interface(CephContext*, std::shared_ptr<DPDKDevice>, EventCenter*)::<lambda()> >::_M_invoke(const std::_Any_data &) (
_functor=...) at /usr/include/c++/9/bits/std_function.h:286
No locals.
#16 0x0000aaaaac36a6c4 in std::function<Tub<Packet> ()>::operator()() const (this=0xaaaab621b6a0)
at /usr/include/c++/9/bits/std_function.h:685
No locals.
#17 DPDKQueuePair::poll_tx (this=0xaaaab5608f00) at ./src/msg/async/dpdk/DPDK.cc:657
p = {object = 0xffffbd3dccf8, raw = "\000\000\000\000\000\000\000", occupied = false}
pr = @0xaaaab621b6a0: {<std::_Maybe_unary_or_binary_function<Tub<Packet> >> = {<No data fields>}, <std::_Function_base> = {static _M_max_size = 16, static _M_max_align = 8, _M_functor = {_M_unused = {_M_object = 0xaaaab67d0008,
_M_const_object = 0xaaaab67d0008, _M_function_pointer = 0xaaaab67d0008,
_M_member_pointer = (void (std::_Undefined_class::*)(std::_Undefined_class * const)) 0xaaaab67d0008, this a---Type <return> to continue, or q <return> to quit---
djustment 1}, _M_pod_data = "\b\000}\266\252\252\000\000\002\000\000\000\000\000\000"},
_M_manager = 0xaaaaac353378 <std::_Function_base::_Base_manager<interface::interface(CephContext*, std::shared_ptr<DPDKDevice>, EventCenter*)::<lambda()> >::_M_manager(std::_Any_data &, const std::_Any_data &, std::_Manager_operation)>},
_M_invoker = 0xaaaaac354290 <std::_Function_handler<Tub<Packet>(), interface::interface(CephContext*, std::shared_ptr<DPDKDevice>, EventCenter*)::<lambda()> >::_M_invoke(const std::_Any_data &)>}
__for_range = std::vector of length 1, capacity 1 = { {<std::_Maybe_unary_or_binary_function<Tub<Packet> >> = {<No data fields>}, <std::_Function_base> = {
static M_max_size = 16, static _M_max_align = 8, _M_functor = {_M_unused = {_M_object = 0xaaaab67d0008,
_M_const_object = 0xaaaab67d0008, _M_function_pointer = 0xaaaab67d0008,
_M_member_pointer = (void (std::_Undefined_class::*)(std::_Undefined_class * const)) 0xaaaab67d0008, this adjustment 1}, _M_pod_data = "\b\000}\266\252\252\000\000\002\000\000\000\000\000\000"},
_M_manager = 0xaaaaac353378 <std::_Function_base::_Base_manager<interface::interface(CephContext*, std::shared_ptr<DPDKDevice>, EventCenter*)::<lambda()> >::_M_manager(std::_Any_data &, const std::_Any_data &, std::_Manager_operation)>},
_M_invoker = 0xaaaaac354290 <std::_Function_handler<Tub<Packet>(), interface::interface(CephContext*, std::shared_ptr<DPDKDevice>, EventCenter*)::<lambda()> >::_M_invoke(const std::_Any_data &)>}}
for_begin = <optimized out>
__for_end = <optimized out>
work = 0
nonloopback = <optimized out>
total_work = 2
#18 0x0000aaaaac3469f0 in DPDKQueuePair::DPDKTXPoller::poll (this=<optimized out>) at ./src/msg/async/dpdk/DPDK.h:675
No locals.
#19 0x0000aaaaabcc65d0 in EventCenter::process_events (this=this@entry=0xaaaab5579048,
timeout_microseconds=<optimized out>, working_dur=working_dur@entry=0xffffbd3dd010)
at /usr/include/c++/9/bits/stl_vector.h:1040
i = 0
tv = {tv_sec = 0, tv_usec = 0}
numevents = 0
trigger_time = true
now = <optimized out>
end_time = <optimized out>
it = <optimized out>
blocking = <optimized out>
__func
= "process_events"
fired_events = std::vector of length 0, capacity 0
working_start = {
_d = {__r = 335540548917520}}
---Type <return> to continue, or q <return> to quit---


Files

osdcrash.txt (288 KB) osdcrash.txt chunsong feng, 11/14/2019 07:40 AM
Actions #1

Updated by Kefu Chai over 4 years ago

  • Status changed from New to Fix Under Review
  • Assignee set to chunsong feng
  • Pull request ID set to 31876
Actions #2

Updated by Kefu Chai over 4 years ago

  • Status changed from Fix Under Review to Resolved
Actions

Also available in: Atom PDF