Bug #39059
assert in ceph::net::SocketMessenger::unregister_conn()
Status:
Can't reproduce
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
INFO 2019-04-01 18:21:13,299 [shard 0] osd - add_peer(4) WARN 2019-04-01 18:21:21,372 [shard 0] ms - [osd.2(hb_front) v1:192.168.2.8:6813/54788 >> v1:192.168.2.8:6817/54927@57912] open fault: std::system_error (error ceph::net:4, read eof) WARN 2019-04-01 18:21:21,372 [shard 0] ms - [osd.2(hb_front) v1:192.168.2.8:6813/54788@59520 >> v1:192.168.2.8:6817/54927] open fault: std::system_error (error ceph::net:4, read eof) WARN 2019-04-01 18:21:21,372 [shard 0] ms - [osd.2(hb_back) v1:192.168.2.8:6812/54788@58978 >> v1:192.168.2.8:6816/54927] open fault: std::system_error (error ceph::net:4, read eof) ERROR 2019-04-01 18:21:21,372 [shard 0] none - ../src/crimson/net/SocketMessenger.cc:285 : In function 'void ceph::net::SocketMessenger::unregister_conn(ceph::net::SocketConnectionRef)', ceph_assert(%s) found != connections.end() Aborting on shard 0. Backtrace: 0x00000000009bb628 0x000000000092ab10 0x000000000092adfd 0x000000000092aec2 0x000029b7ea04872f /lib/x86_64-linux-gnu/libc.so.6+0x00000000000378ba /lib/x86_64-linux-gnu/libc.so.6+0x0000000000022534 0x00000000002d469e 0x00000000002d46b2 0x00000000007dda99 0x000000000083b21e 0x000000000083bd9f 0x00000000002f573d 0x00000000002fb94f 0x00000000009283e3 0x000000000092872f 0x000000000099f1e5 0x00000000008e7777 0x000000000032bebd /lib/x86_64-linux-gnu/libc.so.6+0x000000000002409a 0x0000000000336fc9 Aborted ...
../src/seastar/scripts/seastar-addr2line -e bin/crimson-osd 0x00000000002d469e 0x00000000002d46b2 0x00000000007dda99 0x000000000083b21e 0x000000000083bd9f 0x00000000002f573d 0x00000000002fb94f 0x00000000009283e3 0x000000000092872f 0x000000000099f1e5 0x00000000008e7777 0x000000000032bebd [Backtrace #0] ceph::__ceph_assert_fail(char const*, char const*, int, char const*) at /var/ssd/ceph/build/../src/crimson/common/assert.cc:25 ceph::__ceph_assert_fail(ceph::assert_data const&) at /var/ssd/ceph/build/../src/crimson/common/assert.cc:13 ceph::net::SocketMessenger::unregister_conn(seastar::shared_ptr<ceph::net::SocketConnection>) at /var/ssd/ceph/build/../src/crimson/net/SocketMessenger.cc:285 (discriminator 1) ceph::net::SocketConnection::do_close() at /var/ssd/ceph/build/../src/crimson/net/SocketConnection.cc:338 operator() at /var/ssd/ceph/build/../src/crimson/net/SocketConnection.cc:106 (inlined by) ?? at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:1486 (inlined by) ?? at /var/ssd/ceph/build/../src/seastar/include/seastar/core/reactor.hh:879 (inlined by) ceph::net::SocketConnection::close() at /var/ssd/ceph/build/../src/crimson/net/SocketConnection.cc:107 operator() at /var/ssd/ceph/build/../src/crimson/net/SocketConnection.cc:924 (inlined by) ?? at /var/ssd/ceph/build/../src/seastar/include/seastar/core/apply.hh:35 (inlined by) ?? at /var/ssd/ceph/build/../src/seastar/include/seastar/core/apply.hh:43 (inlined by) ?? at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:1430 (inlined by) ?? at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:1464 (inlined by) ?? at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:1004 (inlined by) ?? at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:985 (inlined by) operator() at /var/ssd/ceph/build/../src/crimson/net/SocketConnection.cc:930 (inlined by) apply<ceph::net::SocketConnection::execute_open()::<lambda()>::<lambda(const std::system_error&)>&, const std::system_error&> at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:1486 operator()<seastar::future<> > at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:1243 (inlined by) ?? at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:1486 (inlined by) operator()<seastar::future_state<> > at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:1069 (inlined by) run_and_dispose at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:425 seastar::reactor::run_tasks(seastar::reactor::task_queue&) at /var/ssd/ceph/build/../src/seastar/src/core/reactor.cc:3590 seastar::reactor::run_some_tasks() [clone .part.0] at /var/ssd/ceph/build/../src/seastar/src/core/reactor.cc:4015 seastar::reactor::run() at /var/ssd/ceph/build/../src/seastar/src/core/reactor.cc:3998 (inlined by) seastar::reactor::run() at /var/ssd/ceph/build/../src/seastar/src/core/reactor.cc:4158 seastar::app_template::run_deprecated(int, char**, std::function<void ()>&&) at /var/ssd/ceph/build/../src/seastar/src/core/app-template.cc:186 main at /var/ssd/ceph/build/../src/crimson/osd/main.cc:95
reproduce steps
MDS=0 MGR=1 OSD=8 MON=1 ../src/vstart.sh -n --without-dashboard --memstore -X bin/ceph osd pool create test-pool 512 512 bin/ceph osd pool set test-pool min_size 1 bin/ceph osd pool set test-pool size 1 bin/rados bench -p test-pool 2 write --no-cleanup for cpu in `seq 0 8`; do bin/init-ceph stop osd.2 bin/crimson-osd -i $cpu -c ceph.conf --smp 1 --cpuset $cpu & done
History
#1 Updated by Kefu Chai over 4 years ago
- Status changed from New to Can't reproduce
not reproducible anymore.