Project

General

Profile

Actions

Bug #47324

closed

unittest_seastar_messenger fails in "make check" run randomly

Added by Kefu Chai over 3 years ago. Updated over 3 years ago.

Status:
Won't Fix
Priority:
Normal
Assignee:
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

79/205 Test #194: unittest_seastar_messenger ................***Exception: Child aborted 24.30 sec
WARNING: debug mode. Not for benchmarking or production
WARN  2020-09-06 08:00:41,223 [shard 0] seastar - Unable to set SCHED_FIFO scheduling policy for timer thread; latency impact possible. Try adding CAP_SYS_NICE
INFO  2020-09-06 08:00:41,742 [shard 0] ms - test_echo(rounds=512, keepalive_ratio=0.1, v2=0):
INFO  2020-09-06 08:00:41,782 [shard 0] ms - [osd.1(server2) v1:127.0.0.1:9011/2] bind: done
INFO  2020-09-06 08:00:41,783 [shard 0] ms - [osd.0(server1) v1:127.0.0.1:9010/1] bind: done
INFO  2020-09-06 08:00:41,784 [shard 0] ms - [osd.3(client2) - >> osd.? v1:127.0.0.1:9010/1] write_event: delay ...
INFO  2020-09-06 08:00:41,784 [shard 0] ms - [osd.2(client1) - >> osd.? v1:127.0.0.1:9011/2] write_event: delay ...
/home/jenkins-build/build/workspace/ceph-pull-requests/src/common/buffer.cc:2154:24: runtime error: member access within null pointer of type 'struct raw'
INFO  2020-09-06 08:00:41,834 [shard 0] ms - [osd.2(client1) 127.0.0.1:0/3@61026 >> osd.? v1:127.0.0.1:9011/2] learned myaddr=127.0.0.1:0/3 (unbound) from v1:127.0.0.1:61026/0
INFO  2020-09-06 08:00:41,834 [shard 0] ms - [osd.3(client2) 127.0.0.1:0/4@50272 >> osd.? v1:127.0.0.1:9010/1] learned myaddr=127.0.0.1:0/4 (unbound) from v1:127.0.0.1:50272/0
INFO  2020-09-06 08:00:46,068 [shard 0] ms - [osd.3(client2) 127.0.0.1:0/4@50272 >> osd.? v1:127.0.0.1:9010/1]: finished sending 512 pings with 54 keepalives
INFO  2020-09-06 08:00:46,095 [shard 0] ms - [osd.2(client1) 127.0.0.1:0/3@61026 >> osd.? v1:127.0.0.1:9011/2]: finished sending 512 pings with 53 keepalives
INFO  2020-09-06 08:00:57,905 [shard 0] ms - [osd.3(client2) 127.0.0.1:0/4@50272 >> osd.? v1:127.0.0.1:9010/1]: finished receiving 512 pongs
INFO  2020-09-06 08:00:57,906 [shard 0] ms - [osd.3(client2) 127.0.0.1:0/4@50272 >> osd.? v1:127.0.0.1:9010/1]: handshake 0.113787, pingpong 16.0086
INFO  2020-09-06 08:00:57,925 [shard 0] ms - [osd.2(client1) 127.0.0.1:0/3@61026 >> osd.? v1:127.0.0.1:9011/2]: finished receiving 512 pongs
INFO  2020-09-06 08:00:57,925 [shard 0] ms - [osd.2(client1) 127.0.0.1:0/3@61026 >> osd.? v1:127.0.0.1:9011/2]: handshake 0.113773, pingpong 16.0274
INFO  2020-09-06 08:00:57,926 [shard 0] ms - client1 shutdown...
INFO  2020-09-06 08:00:57,927 [shard 0] ms - [osd.2(client1) 127.0.0.1:0/3@61026 >> osd.? v1:127.0.0.1:9011/2] closing: reset no, replace no
WARN  2020-09-06 08:00:57,929 [shard 0] ms - [osd.1(server2) v1:127.0.0.1:9011/2 >> osd.? v1:127.0.0.1:0/3@61026] open fault: std::system_error (error crimson::net:4, read eof)
INFO  2020-09-06 08:00:57,929 [shard 0] ms - [osd.1(server2) v1:127.0.0.1:9011/2 >> osd.? v1:127.0.0.1:0/3@61026] closing: reset yes, replace no
WARN  2020-09-06 08:00:57,930 [shard 0] ms - [osd.2(client1) 127.0.0.1:0/3@61026 >> osd.? v1:127.0.0.1:9011/2] open fault: std::system_error (error crimson::net:4, read eof)
INFO  2020-09-06 08:00:57,934 [shard 0] ms - client2 shutdown...
INFO  2020-09-06 08:00:57,934 [shard 0] ms - [osd.3(client2) 127.0.0.1:0/4@50272 >> osd.? v1:127.0.0.1:9010/1] closing: reset no, replace no
WARN  2020-09-06 08:00:57,936 [shard 0] ms - [osd.3(client2) 127.0.0.1:0/4@50272 >> osd.? v1:127.0.0.1:9010/1] open fault: std::system_error (error crimson::net:4, read eof)
WARN  2020-09-06 08:00:57,937 [shard 0] ms - [osd.0(server1) v1:127.0.0.1:9010/1 >> osd.? v1:127.0.0.1:0/4@50272] open fault: std::system_error (error crimson::net:4, read eof)
INFO  2020-09-06 08:00:57,937 [shard 0] ms - [osd.0(server1) v1:127.0.0.1:9010/1 >> osd.? v1:127.0.0.1:0/4@50272] closing: reset yes, replace no
INFO  2020-09-06 08:00:57,940 [shard 0] ms - server1 shutdown...
INFO  2020-09-06 08:00:57,944 [shard 0] ms - server2 shutdown...
INFO  2020-09-06 08:00:57,947 [shard 0] ms - test_echo() done!

INFO  2020-09-06 08:00:57,947 [shard 0] ms - test_echo(rounds=512, keepalive_ratio=0.1, v2=1):
INFO  2020-09-06 08:00:57,951 [shard 0] ms - [osd.1(server2) v2:127.0.0.1:9011/2] bind: done
INFO  2020-09-06 08:00:57,952 [shard 0] ms - [osd.0(server1) v2:127.0.0.1:9010/1] bind: done
INFO  2020-09-06 08:00:57,953 [shard 0] ms - [osd.3(client2) - >> osd.? v2:127.0.0.1:9010/1] ProtocolV2::start_connect(): peer_addr=v2:127.0.0.1:9010/1, peer_name=osd.?, cc=16450192496884118844 policy(lossy=1, server=0, standby=0, resetcheck=0)
INFO  2020-09-06 08:00:57,954 [shard 0] ms - [osd.3(client2) - >> osd.? v2:127.0.0.1:9010/1] write_event: delay ...
INFO  2020-09-06 08:00:57,955 [shard 0] ms - [osd.2(client1) - >> osd.? v2:127.0.0.1:9011/2] ProtocolV2::start_connect(): peer_addr=v2:127.0.0.1:9011/2, peer_name=osd.?, cc=8039375379809172431 policy(lossy=1, server=0, standby=0, resetcheck=0)
INFO  2020-09-06 08:00:57,956 [shard 0] ms - [osd.2(client1) - >> osd.? v2:127.0.0.1:9011/2] write_event: delay ...
INFO  2020-09-06 08:00:57,960 [shard 0] ms - [osd.1(server2) v2:127.0.0.1:9011/2 >> unknown.? -@60489] ProtocolV2::start_accept(): target_addr=127.0.0.1:60489/0
INFO  2020-09-06 08:00:57,962 [shard 0] ms - [osd.0(server1) v2:127.0.0.1:9010/1 >> unknown.? -@63633] ProtocolV2::start_accept(): target_addr=127.0.0.1:63633/0
INFO  2020-09-06 08:00:57,995 [shard 0] ms - [osd.0(server1) v2:127.0.0.1:9010/1 >> osd.? -@63633] UPDATE: peer_type=osd, policy(lossy=1 server=1 standby=0 resetcheck=0)
INFO  2020-09-06 08:00:57,997 [shard 0] ms - [osd.3(client2) 127.0.0.1:0/4@63633 >> osd.? v2:127.0.0.1:9010/1] learned myaddr=127.0.0.1:0/4 (unbound) from v2:127.0.0.1:63633/0
INFO  2020-09-06 08:00:58,002 [shard 0] ms - [osd.1(server2) v2:127.0.0.1:9011/2 >> osd.? -@60489] UPDATE: peer_type=osd, policy(lossy=1 server=1 standby=0 resetcheck=0)
INFO  2020-09-06 08:00:58,003 [shard 0] ms - [osd.2(client1) 127.0.0.1:0/3@60489 >> osd.? v2:127.0.0.1:9011/2] learned myaddr=127.0.0.1:0/3 (unbound) from v2:127.0.0.1:60489/0
INFO  2020-09-06 08:00:58,043 [shard 0] ms - [osd.0(server1) v2:127.0.0.1:9010/1 >> osd.3 127.0.0.1:0/4@63633] established: gs=0, pgs=1, cs=0, client_cookie=16450192496884118844, server_cookie=0, in_seq=0, out_seq=0, out_q=0
INFO  2020-09-06 08:00:58,054 [shard 0] ms - [osd.3(client2) 127.0.0.1:0/4@63633 >> osd.0 v2:127.0.0.1:9010/1] connected: gs=1, pgs=1, cs=0, client_cookie=16450192496884118844, server_cookie=0, in_seq=0, out_seq=0, out_q=59
INFO  2020-09-06 08:00:58,106 [shard 0] ms - [osd.1(server2) v2:127.0.0.1:9011/2 >> osd.2 127.0.0.1:0/3@60489] established: gs=0, pgs=1, cs=0, client_cookie=8039375379809172431, server_cookie=0, in_seq=0, out_seq=0, out_q=0
INFO  2020-09-06 08:00:58,123 [shard 0] ms - [osd.2(client1) 127.0.0.1:0/3@60489 >> osd.1 v2:127.0.0.1:9011/2] connected: gs=1, pgs=1, cs=0, client_cookie=8039375379809172431, server_cookie=0, in_seq=0, out_seq=0, out_q=70
INFO  2020-09-06 08:00:59,870 [shard 0] ms - [osd.2(client1) 127.0.0.1:0/3@60489 >> osd.1 v2:127.0.0.1:9011/2]: finished sending 512 pings with 63 keepalives
INFO  2020-09-06 08:00:59,911 [shard 0] ms - [osd.3(client2) 127.0.0.1:0/4@63633 >> osd.0 v2:127.0.0.1:9010/1]: finished sending 512 pings with 47 keepalives
INFO  2020-09-06 08:01:03,783 [shard 0] ms - [osd.2(client1) 127.0.0.1:0/3@60489 >> osd.1 v2:127.0.0.1:9011/2]: finished receiving 512 pongs
INFO  2020-09-06 08:01:03,784 [shard 0] ms - [osd.2(client1) 127.0.0.1:0/3@60489 >> osd.1 v2:127.0.0.1:9011/2]: handshake 0.168544, pingpong 5.66031
INFO  2020-09-06 08:01:03,788 [shard 0] ms - [osd.3(client2) 127.0.0.1:0/4@63633 >> osd.0 v2:127.0.0.1:9010/1]: finished receiving 512 pongs
INFO  2020-09-06 08:01:03,789 [shard 0] ms - [osd.3(client2) 127.0.0.1:0/4@63633 >> osd.0 v2:127.0.0.1:9010/1]: handshake 0.102047, pingpong 5.73343
INFO  2020-09-06 08:01:03,789 [shard 0] ms - client1 shutdown...
INFO  2020-09-06 08:01:03,790 [shard 0] ms - [osd.2(client1) 127.0.0.1:0/3@60489 >> osd.1 v2:127.0.0.1:9011/2] closing: reset no, replace no
INFO  2020-09-06 08:01:03,791 [shard 0] ms - [osd.2(client1) 127.0.0.1:0/3@60489 >> osd.1 v2:127.0.0.1:9011/2] execute_ready(): protocol aborted at CLOSING -- std::system_error (error crimson::net:4, read eof)
INFO  2020-09-06 08:01:03,792 [shard 0] ms - [osd.1(server2) v2:127.0.0.1:9011/2 >> osd.2 127.0.0.1:0/3@60489] execute_ready(): fault at READY on lossy channel, going to CLOSING -- std::system_error (error crimson::net:4, read eof)
INFO  2020-09-06 08:01:03,792 [shard 0] ms - [osd.1(server2) v2:127.0.0.1:9011/2 >> osd.2 127.0.0.1:0/3@60489] closing: reset yes, replace no
INFO  2020-09-06 08:01:03,794 [shard 0] ms - client2 shutdown...
INFO  2020-09-06 08:01:03,794 [shard 0] ms - [osd.3(client2) 127.0.0.1:0/4@63633 >> osd.0 v2:127.0.0.1:9010/1] closing: reset no, replace no
INFO  2020-09-06 08:01:03,796 [shard 0] ms - [osd.3(client2) 127.0.0.1:0/4@63633 >> osd.0 v2:127.0.0.1:9010/1] execute_ready(): protocol aborted at CLOSING -- std::system_error (error crimson::net:4, read eof)
INFO  2020-09-06 08:01:03,796 [shard 0] ms - [osd.0(server1) v2:127.0.0.1:9010/1 >> osd.3 127.0.0.1:0/4@63633] execute_ready(): fault at READY on lossy channel, going to CLOSING -- std::system_error (error crimson::net:4, read eof)
INFO  2020-09-06 08:01:03,796 [shard 0] ms - [osd.0(server1) v2:127.0.0.1:9010/1 >> osd.3 127.0.0.1:0/4@63633] closing: reset yes, replace no
INFO  2020-09-06 08:01:03,799 [shard 0] ms - server1 shutdown...
INFO  2020-09-06 08:01:03,801 [shard 0] ms - server2 shutdown...
INFO  2020-09-06 08:01:03,803 [shard 0] ms - test_echo() done!

INFO  2020-09-06 08:01:03,804 [shard 0] ms - test_concurrent_dispatch(v2=0):
INFO  2020-09-06 08:01:03,805 [shard 0] ms - [osd.4(server3) v1:127.0.0.1:9010/5] bind: done
INFO  2020-09-06 08:01:03,807 [shard 0] ms - [osd.5(client3) - >> osd.? v1:127.0.0.1:9010/5] write_event: delay ...
INFO  2020-09-06 08:01:03,811 [shard 0] ms - [osd.5(client3) 127.0.0.1:0/6@59327 >> osd.? v1:127.0.0.1:9010/5] learned myaddr=127.0.0.1:0/6 (unbound) from v1:127.0.0.1:59327/0
INFO  2020-09-06 08:01:03,823 [shard 0] ms - client shutdown...
INFO  2020-09-06 08:01:03,824 [shard 0] ms - [osd.5(client3) 127.0.0.1:0/6@59327 >> osd.? v1:127.0.0.1:9010/5] closing: reset no, replace no
WARN  2020-09-06 08:01:03,825 [shard 0] ms - [osd.4(server3) v1:127.0.0.1:9010/5 >> osd.? v1:127.0.0.1:0/6@59327] open fault: std::system_error (error crimson::net:4, read eof)
INFO  2020-09-06 08:01:03,825 [shard 0] ms - [osd.4(server3) v1:127.0.0.1:9010/5 >> osd.? v1:127.0.0.1:0/6@59327] closing: reset yes, replace no
WARN  2020-09-06 08:01:03,827 [shard 0] ms - [osd.5(client3) 127.0.0.1:0/6@59327 >> osd.? v1:127.0.0.1:9010/5] open fault: std::system_error (error crimson::net:4, read eof)
INFO  2020-09-06 08:01:03,829 [shard 0] ms - server shutdown...
INFO  2020-09-06 08:01:03,831 [shard 0] ms - test_concurrent_dispatch() done!

INFO  2020-09-06 08:01:03,831 [shard 0] ms - test_concurrent_dispatch(v2=1):
INFO  2020-09-06 08:01:03,832 [shard 0] ms - [osd.4(server3) v2:127.0.0.1:9010/5] bind: done
INFO  2020-09-06 08:01:03,833 [shard 0] ms - [osd.5(client3) - >> osd.? v2:127.0.0.1:9010/5] ProtocolV2::start_connect(): peer_addr=v2:127.0.0.1:9010/5, peer_name=osd.?, cc=14102231378407053329 policy(lossy=1, server=0, standby=0, resetcheck=0)
INFO  2020-09-06 08:01:03,834 [shard 0] ms - [osd.5(client3) - >> osd.? v2:127.0.0.1:9010/5] write_event: delay ...
INFO  2020-09-06 08:01:03,835 [shard 0] ms - [osd.4(server3) v2:127.0.0.1:9010/5 >> unknown.? -@55488] ProtocolV2::start_accept(): target_addr=127.0.0.1:55488/0
INFO  2020-09-06 08:01:03,846 [shard 0] ms - [osd.4(server3) v2:127.0.0.1:9010/5 >> osd.? -@55488] UPDATE: peer_type=osd, policy(lossy=1 server=1 standby=0 resetcheck=0)
INFO  2020-09-06 08:01:03,847 [shard 0] ms - [osd.5(client3) 127.0.0.1:0/6@55488 >> osd.? v2:127.0.0.1:9010/5] learned myaddr=127.0.0.1:0/6 (unbound) from v2:127.0.0.1:55488/0
INFO  2020-09-06 08:01:03,860 [shard 0] ms - [osd.4(server3) v2:127.0.0.1:9010/5 >> osd.5 127.0.0.1:0/6@55488] established: gs=0, pgs=1, cs=0, client_cookie=14102231378407053329, server_cookie=0, in_seq=0, out_seq=0, out_q=0
INFO  2020-09-06 08:01:03,862 [shard 0] ms - [osd.5(client3) 127.0.0.1:0/6@55488 >> osd.4 v2:127.0.0.1:9010/5] connected: gs=1, pgs=1, cs=0, client_cookie=14102231378407053329, server_cookie=0, in_seq=0, out_seq=0, out_q=2
INFO  2020-09-06 08:01:03,868 [shard 0] ms - client shutdown...
INFO  2020-09-06 08:01:03,868 [shard 0] ms - [osd.5(client3) 127.0.0.1:0/6@55488 >> osd.4 v2:127.0.0.1:9010/5] closing: reset no, replace no
INFO  2020-09-06 08:01:03,870 [shard 0] ms - [osd.4(server3) v2:127.0.0.1:9010/5 >> osd.5 127.0.0.1:0/6@55488] execute_ready(): fault at READY on lossy channel, going to CLOSING -- std::system_error (error crimson::net:4, read eof)
INFO  2020-09-06 08:01:03,870 [shard 0] ms - [osd.4(server3) v2:127.0.0.1:9010/5 >> osd.5 127.0.0.1:0/6@55488] closing: reset yes, replace no
INFO  2020-09-06 08:01:03,871 [shard 0] ms - [osd.5(client3) 127.0.0.1:0/6@55488 >> osd.4 v2:127.0.0.1:9010/5] execute_ready(): protocol aborted at CLOSING -- std::system_error (error crimson::net:4, read eof)
INFO  2020-09-06 08:01:03,873 [shard 0] ms - server shutdown...
INFO  2020-09-06 08:01:03,875 [shard 0] ms - test_concurrent_dispatch() done!

INFO  2020-09-06 08:01:03,876 [shard 0] ms - test_preemptive_shutdown(v2=0):
INFO  2020-09-06 08:01:03,877 [shard 0] ms - [osd.6(server4) v1:127.0.0.1:9010/7] bind: done
INFO  2020-09-06 08:01:03,878 [shard 0] ms - [osd.7(client4) - >> osd.? v1:127.0.0.1:9010/7] write_event: delay ...
INFO  2020-09-06 08:01:03,884 [shard 0] ms - [osd.7(client4) 127.0.0.1:0/8@53985 >> osd.? v1:127.0.0.1:9010/7] learned myaddr=127.0.0.1:0/8 (unbound) from v1:127.0.0.1:53985/0
INFO  2020-09-06 08:01:03,980 [shard 0] ms - client shutdown...
INFO  2020-09-06 08:01:03,981 [shard 0] ms - [osd.7(client4) 127.0.0.1:0/8@53985 >> osd.? v1:127.0.0.1:9010/7] closing: reset no, replace no
WARN  2020-09-06 08:01:03,988 [shard 0] ms - [osd.7(client4) 127.0.0.1:0/8@53985 >> osd.? v1:127.0.0.1:9010/7] open fault: seastar::gate_closed_exception (gate closed)
INFO  2020-09-06 08:01:03,992 [shard 0] ms - server shutdown...
INFO  2020-09-06 08:01:03,996 [shard 0] ms - [osd.6(server4) v1:127.0.0.1:9010/7 >> osd.? v1:127.0.0.1:0/8@53985] closing: reset no, replace no
WARN  2020-09-06 08:01:03,998 [shard 0] ms - [osd.6(server4) v1:127.0.0.1:9010/7 >> osd.? v1:127.0.0.1:0/8@53985] open fault: seastar::gate_closed_exception (gate closed)
INFO  2020-09-06 08:01:04,000 [shard 0] ms - test_preemptive_shutdown() done!

INFO  2020-09-06 08:01:04,000 [shard 0] ms - test_preemptive_shutdown(v2=1):
INFO  2020-09-06 08:01:04,001 [shard 0] ms - [osd.6(server4) v2:127.0.0.1:9010/7] bind: done
INFO  2020-09-06 08:01:04,002 [shard 0] ms - [osd.7(client4) - >> osd.? v2:127.0.0.1:9010/7] ProtocolV2::start_connect(): peer_addr=v2:127.0.0.1:9010/7, peer_name=osd.?, cc=8819170180176621360 policy(lossy=1, server=0, standby=0, resetcheck=0)
INFO  2020-09-06 08:01:04,003 [shard 0] ms - [osd.7(client4) - >> osd.? v2:127.0.0.1:9010/7] write_event: delay ...
INFO  2020-09-06 08:01:04,005 [shard 0] ms - [osd.6(server4) v2:127.0.0.1:9010/7 >> unknown.? -@56063] ProtocolV2::start_accept(): target_addr=127.0.0.1:56063/0
INFO  2020-09-06 08:01:04,020 [shard 0] ms - [osd.6(server4) v2:127.0.0.1:9010/7 >> osd.? -@56063] UPDATE: peer_type=osd, policy(lossy=1 server=1 standby=0 resetcheck=0)
INFO  2020-09-06 08:01:04,021 [shard 0] ms - [osd.7(client4) 127.0.0.1:0/8@56063 >> osd.? v2:127.0.0.1:9010/7] learned myaddr=127.0.0.1:0/8 (unbound) from v2:127.0.0.1:56063/0
INFO  2020-09-06 08:01:04,082 [shard 0] ms - [osd.6(server4) v2:127.0.0.1:9010/7 >> osd.7 127.0.0.1:0/8@56063] established: gs=0, pgs=1, cs=0, client_cookie=8819170180176621360, server_cookie=0, in_seq=0, out_seq=0, out_q=0
INFO  2020-09-06 08:01:04,087 [shard 0] ms - [osd.7(client4) 127.0.0.1:0/8@56063 >> osd.6 v2:127.0.0.1:9010/7] connected: gs=1, pgs=1, cs=0, client_cookie=8819170180176621360, server_cookie=0, in_seq=0, out_seq=0, out_q=258
INFO  2020-09-06 08:01:04,120 [shard 0] ms - client shutdown...
INFO  2020-09-06 08:01:04,120 [shard 0] ms - [osd.7(client4) 127.0.0.1:0/8@56063 >> osd.6 v2:127.0.0.1:9010/7] closing: reset no, replace no
INFO  2020-09-06 08:01:04,124 [shard 0] ms - [osd.6(server4) v2:127.0.0.1:9010/7 >> osd.7 127.0.0.1:0/8@56063] execute_ready(): fault at READY on lossy channel, going to CLOSING -- std::system_error (error crimson::net:4, read eof)
INFO  2020-09-06 08:01:04,124 [shard 0] ms - [osd.6(server4) v2:127.0.0.1:9010/7 >> osd.7 127.0.0.1:0/8@56063] closing: reset yes, replace no
INFO  2020-09-06 08:01:04,125 [shard 0] ms - [osd.7(client4) 127.0.0.1:0/8@56063 >> osd.6 v2:127.0.0.1:9010/7] execute_ready(): protocol aborted at CLOSING -- std::system_error (error crimson::net:4, read eof)
INFO  2020-09-06 08:01:04,133 [shard 0] ms - server shutdown...
INFO  2020-09-06 08:01:04,136 [shard 0] ms - test_preemptive_shutdown() done!

INFO  2020-09-06 08:01:04,136 [shard 0] ms - test_v2_protocol: start local TestPeer at v2:127.0.0.1:9013/0...
ERROR 2020-09-06 08:01:04,139 [shard 0] none - /home/jenkins-build/build/workspace/ceph-pull-requests/src/crimson/net/SocketMessenger.h:65 : In function 'virtual crimson::net::SocketMessenger::~SocketMessenger()', ceph_assert(%s)
!listener
Aborting on shard 0.
Backtrace:
  /usr/lib/x86_64-linux-gnu/libasan.so.5+0x000000000006bb2f
  0x000000000c7d20b8
  0x000000000c7aaa27
  0x000000000c66163e
  0x000000000c6617b1
  0x000000000c6c3945
  0x000000000c6fbfc6
  0x000000000c6fc061
  /lib/x86_64-linux-gnu/libpthread.so.0+0x000000000001288f
  /lib/x86_64-linux-gnu/libc.so.6+0x000000000003ee96
  /lib/x86_64-linux-gnu/libc.so.6+0x0000000000040800
  0x000000000bea23c8
  0x000000000a3bca87
  0x000000000a3c35cf
  0x000000000a3c3667
  0x000000000a236ed3
  0x000000000a0556ec
  0x000000000a0557fd
  0x000000000a055907
  0x0000000009ff1807
  0x0000000009f6de9b
  0x0000000009ff1b99
  0x000000000a0e0cdd
  0x000000000a055e2d
  0x000000000a1b7ba2
  0x000000000a1b7e3f
  0x000000000c6a7d84
  0x000000000c6ac312
  0x000000000c6b183c
  0x000000000c52514e
  0x000000000c522fc5
  0x000000000c523565
  0x0000000009fd0f79
  /lib/x86_64-linux-gnu/libc.so.6+0x0000000000021b96
  0x0000000009f2bae9

Related issues 1 (0 open1 closed)

Related to crimson - Bug #48108: unittest_seastar_messenger failsResolvedYingxin Cheng

Actions
Actions #1

Updated by Kefu Chai over 3 years ago

[Backtrace #0]
__interceptor_tcgetattr at ??:?
void seastar::backtrace<seastar::backtrace_buffer::append_backtrace()::{lambda(seastar::frame)#1}>(seastar::backtrace_buffer::append_backtrace()::{lambda(seastar::frame)#1}&&) at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/include/seastar/util/backtrace.hh:59
seastar::backtrace_buffer::append_backtrace() at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/src/core/reactor.cc:730
seastar::print_with_backtrace(seastar::backtrace_buffer&) at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/src/core/reactor.cc:751
seastar::print_with_backtrace(char const*) at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/src/core/reactor.cc:758
seastar::sigabrt_action() at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/src/core/reactor.cc:3517
seastar::install_oneshot_signal_handler<6, &seastar::sigabrt_action>()::{lambda(int, siginfo_t*, void*)#1}::operator()(int, siginfo_t*, void*) const at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/src/core/reactor.cc:3499
seastar::install_oneshot_signal_handler<6, &seastar::sigabrt_action>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/src/core/reactor.cc:3495
__funlockfile at ??:?
__libc_signal_restore_set at /build/glibc-OTsEL5/glibc-2.27/signal/../sysdeps/unix/sysv/linux/nptl-signals.h:80 (discriminator 3)
 (inlined by) __GI_raise at /build/glibc-OTsEL5/glibc-2.27/signal/../sysdeps/unix/sysv/linux/raise.c:48 (discriminator 3)
__GI_abort at /build/glibc-OTsEL5/glibc-2.27/stdlib/abort.c:79
ceph::__ceph_assert_fail(char const*, char const*, int, char const*) at /home/jenkins-build/build/workspace/ceph-pull-requests/src/crimson/common/assert.cc:27
crimson::net::SocketMessenger::~SocketMessenger() at /home/jenkins-build/build/workspace/ceph-pull-requests/src/crimson/net/SocketMessenger.h:65 (discriminator 1)
seastar::shared_ptr_count_for<crimson::net::SocketMessenger>::~shared_ptr_count_for() at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/include/seastar/core/shared_ptr.hh:463
seastar::shared_ptr_count_for<crimson::net::SocketMessenger>::~shared_ptr_count_for() at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/include/seastar/core/shared_ptr.hh:463
seastar::shared_ptr<crimson::net::Messenger>::~shared_ptr() at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/include/seastar/core/shared_ptr.hh:535 (discriminator 1)
(anonymous namespace)::FailoverTestPeer::~FailoverTestPeer() at /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/crimson/test_messenger.cc:1514
(anonymous namespace)::FailoverTestPeer::~FailoverTestPeer() at /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/crimson/test_messenger.cc:1514
std::default_delete<(anonymous namespace)::FailoverTestPeer>::operator()((anonymous namespace)::FailoverTestPeer*) const at /usr/include/c++/9/bits/unique_ptr.h:81 (discriminator 1)
std::unique_ptr<(anonymous namespace)::FailoverTestPeer, std::default_delete<(anonymous namespace)::FailoverTestPeer> >::~unique_ptr() at /usr/include/c++/9/bits/unique_ptr.h:284
(anonymous namespace)::FailoverTestPeer::create(entity_addr_t)::{lambda()#1}::~entity_addr_t() at /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/crimson/test_messenger.cc:1625
_ZZN7seastar6futureIJEE4thenIZN12_GLOBAL__N_116FailoverTestPeer6createE13entity_addr_tEUlvE_NS0_IJSt10unique_ptrIS4_St14default_deleteIS4_EEEEEEET0_OT_ENUlDpOT_E_D2Ev at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/include/seastar/core/future.hh:1496
_ZN7seastar20noncopyable_functionIFNS_6futureIJSt10unique_ptrIN12_GLOBAL__N_116FailoverTestPeerESt14default_deleteIS4_EEEEEvEE17direct_vtable_forIZNS1_IJEE4thenIZNS4_6createE13entity_addr_tEUlvE_S8_EET0_OT_EUlDpOT_E_E7destroyEPNS_8internal25noncopyable_function_baseE at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/include/seastar/util/noncopyable_function.hh:128
seastar::noncopyable_function<seastar::future<std::unique_ptr<(anonymous namespace)::FailoverTestPeer, std::default_delete<(anonymous namespace)::FailoverTestPeer> > > ()>::~noncopyable_function() at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/include/seastar/util/noncopyable_function.hh:181
seastar::continuation<seastar::internal::promise_base_with_type<std::unique_ptr<(anonymous namespace)::FailoverTestPeer, std::default_delete<(anonymous namespace)::FailoverTestPeer> > >, seastar::noncopyable_function<seastar::future<std::unique_ptr<(anonymous namespace)::FailoverTestPeer, std::default_delete<(anonymous namespace)::FailoverTestPeer> > > ()>, seastar::future<>::then_impl_nrvo<seastar::noncopyable_function<seastar::future<std::unique_ptr<(anonymous namespace)::FailoverTestPeer, std::default_delete<(anonymous namespace)::FailoverTestPeer> > > ()>, seastar::future<std::unique_ptr<(anonymous namespace)::FailoverTestPeer, std::default_delete<(anonymous namespace)::FailoverTestPeer> > > >(seastar::noncopyable_function<seastar::future<std::unique_ptr<(anonymous namespace)::FailoverTestPeer, std::default_delete<(anonymous namespace)::FailoverTestPeer> > > ()>&&)::{lambda()#1}::operator()() const::{lambda(seastar::internal::promise_base_with_type<std::unique_ptr<(anonymous namespace)::FailoverTestPeer, std::default_delete<(anonymous namespace)::FailoverTestPeer> > >&, seastar::noncopyable_function<seastar::future<std::unique_ptr<(anonymous namespace)::FailoverTestPeer, std::default_delete<(anonymous namespace)::FailoverTestPeer> > > ()>&, seastar::future_state<>&&)#1}>::~continuation() at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/include/seastar/core/future.hh:631
seastar::continuation<seastar::internal::promise_base_with_type<std::unique_ptr<(anonymous namespace)::FailoverTestPeer, std::default_delete<(anonymous namespace)::FailoverTestPeer> > >, seastar::noncopyable_function<seastar::future<std::unique_ptr<(anonymous namespace)::FailoverTestPeer, std::default_delete<(anonymous namespace)::FailoverTestPeer> > > ()>, seastar::future<>::then_impl_nrvo<seastar::noncopyable_function<seastar::future<std::unique_ptr<(anonymous namespace)::FailoverTestPeer, std::default_delete<(anonymous namespace)::FailoverTestPeer> > > ()>, seastar::future<std::unique_ptr<(anonymous namespace)::FailoverTestPeer, std::default_delete<(anonymous namespace)::FailoverTestPeer> > > >(seastar::noncopyable_function<seastar::future<std::unique_ptr<(anonymous namespace)::FailoverTestPeer, std::default_delete<(anonymous namespace)::FailoverTestPeer> > > ()>&&)::{lambda()#1}::operator()() const::{lambda(seastar::internal::promise_base_with_type<std::unique_ptr<(anonymous namespace)::FailoverTestPeer, std::default_delete<(anonymous namespace)::FailoverTestPeer> > >&, seastar::noncopyable_function<seastar::future<std::unique_ptr<(anonymous namespace)::FailoverTestPeer, std::default_delete<(anonymous namespace)::FailoverTestPeer> > > ()>&, seastar::future_state<>&&)#1}>::run_and_dispose() at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/include/seastar/core/future.hh:652 (discriminator 1)
seastar::reactor::run_tasks(seastar::reactor::task_queue&) at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/src/core/reactor.cc:2192
seastar::reactor::run_some_tasks() at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/src/core/reactor.cc:2608 (discriminator 2)
seastar::reactor::run() at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/src/core/reactor.cc:2763
seastar::app_template::run_deprecated(int, char**, std::function<void ()>&&) at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/src/core/app-template.cc:207 (discriminator 1)
seastar::app_template::run(int, char**, std::function<seastar::future<int> ()>&&) at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/src/core/app-template.cc:115 (discriminator 2)
seastar::app_template::run(int, char**, std::function<seastar::future<> ()>&&) at /home/jenkins-build/build/workspace/ceph-pull-requests/src/seastar/src/core/app-template.cc:130 (discriminator 2)
main at /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/crimson/test_messenger.cc:3570 (discriminator 1)
__libc_start_main at /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
_start at ??:?
Actions #2

Updated by Kefu Chai over 3 years ago

  • Status changed from New to Won't Fix
jenkins-build@braggi14:~/build/workspace/ceph-pull-requests/build$ lsof -i :9013
COMMAND    PID          USER   FD   TYPE    DEVICE SIZE/OFF NODE NAME
unittest_ 1146 jenkins-build    8u  IPv4 618976412      0t0  TCP localhost:9013 (LISTEN)
unittest_ 1146 jenkins-build    9u  IPv4 618976413      0t0  TCP localhost:50595->localhost:9013 (ESTABLISHED)
unittest_ 1146 jenkins-build   10u  IPv4 618976414      0t0  TCP localhost:9013->localhost:50595 (ESTABLISHED)

jenkins-build@braggi14:~/build/workspace/ceph-pull-requests/build$ ps -aux -q 1146
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
jenkins+  1146  1.3  0.2 21475411048 593244 ?  Sl   Sep05  25:23 /home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_seastar_messenger --memory 256M --smp 1

jenkins-build@braggi14:~/build/workspace/ceph-pull-requests/build$ date
Sun Sep  6 10:00:34 UTC 2020
Actions #3

Updated by Kefu Chai over 3 years ago

created https://github.com/ceph/ceph/pull/37326 for better error message.

Actions #4

Updated by Kefu Chai over 3 years ago

  • Related to Bug #48108: unittest_seastar_messenger fails added
Actions

Also available in: Atom PDF