Project

General

Profile

Bug #39059

assert in ceph::net::SocketMessenger::unregister_conn()

Added by Kefu Chai almost 5 years ago. Updated over 4 years ago.

Status:
Can't reproduce
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

INFO  2019-04-01 18:21:13,299 [shard 0] osd - add_peer(4)
WARN  2019-04-01 18:21:21,372 [shard 0] ms - [osd.2(hb_front) v1:192.168.2.8:6813/54788 >> v1:192.168.2.8:6817/54927@57912] open fault: std::system_error (error ceph::net:4, read eof)
WARN  2019-04-01 18:21:21,372 [shard 0] ms - [osd.2(hb_front) v1:192.168.2.8:6813/54788@59520 >> v1:192.168.2.8:6817/54927] open fault: std::system_error (error ceph::net:4, read eof)
WARN  2019-04-01 18:21:21,372 [shard 0] ms - [osd.2(hb_back) v1:192.168.2.8:6812/54788@58978 >> v1:192.168.2.8:6816/54927] open fault: std::system_error (error ceph::net:4, read eof)
ERROR 2019-04-01 18:21:21,372 [shard 0] none - ../src/crimson/net/SocketMessenger.cc:285 : In function 'void ceph::net::SocketMessenger::unregister_conn(ceph::net::SocketConnectionRef)', ceph_assert(%s)
found != connections.end()
Aborting on shard 0.
Backtrace:
  0x00000000009bb628
  0x000000000092ab10
  0x000000000092adfd
  0x000000000092aec2
  0x000029b7ea04872f
  /lib/x86_64-linux-gnu/libc.so.6+0x00000000000378ba
  /lib/x86_64-linux-gnu/libc.so.6+0x0000000000022534
  0x00000000002d469e
  0x00000000002d46b2
  0x00000000007dda99
  0x000000000083b21e
  0x000000000083bd9f
  0x00000000002f573d
  0x00000000002fb94f
  0x00000000009283e3
  0x000000000092872f
  0x000000000099f1e5
  0x00000000008e7777
  0x000000000032bebd
  /lib/x86_64-linux-gnu/libc.so.6+0x000000000002409a
  0x0000000000336fc9
Aborted
...
../src/seastar/scripts/seastar-addr2line -e bin/crimson-osd
0x00000000002d469e
  0x00000000002d46b2
  0x00000000007dda99
  0x000000000083b21e
  0x000000000083bd9f
  0x00000000002f573d
  0x00000000002fb94f
  0x00000000009283e3
  0x000000000092872f
  0x000000000099f1e5
  0x00000000008e7777
  0x000000000032bebd
[Backtrace #0]
ceph::__ceph_assert_fail(char const*, char const*, int, char const*) at /var/ssd/ceph/build/../src/crimson/common/assert.cc:25
ceph::__ceph_assert_fail(ceph::assert_data const&) at /var/ssd/ceph/build/../src/crimson/common/assert.cc:13
ceph::net::SocketMessenger::unregister_conn(seastar::shared_ptr<ceph::net::SocketConnection>) at /var/ssd/ceph/build/../src/crimson/net/SocketMessenger.cc:285 (discriminator 1)
ceph::net::SocketConnection::do_close() at /var/ssd/ceph/build/../src/crimson/net/SocketConnection.cc:338
operator() at /var/ssd/ceph/build/../src/crimson/net/SocketConnection.cc:106
 (inlined by) ?? at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:1486
 (inlined by) ?? at /var/ssd/ceph/build/../src/seastar/include/seastar/core/reactor.hh:879
 (inlined by) ceph::net::SocketConnection::close() at /var/ssd/ceph/build/../src/crimson/net/SocketConnection.cc:107
operator() at /var/ssd/ceph/build/../src/crimson/net/SocketConnection.cc:924
 (inlined by) ?? at /var/ssd/ceph/build/../src/seastar/include/seastar/core/apply.hh:35
 (inlined by) ?? at /var/ssd/ceph/build/../src/seastar/include/seastar/core/apply.hh:43
 (inlined by) ?? at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:1430
 (inlined by) ?? at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:1464
 (inlined by) ?? at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:1004
 (inlined by) ?? at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:985
 (inlined by) operator() at /var/ssd/ceph/build/../src/crimson/net/SocketConnection.cc:930
 (inlined by) apply<ceph::net::SocketConnection::execute_open()::<lambda()>::<lambda(const std::system_error&)>&, const std::system_error&> at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:1486
operator()<seastar::future<> > at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:1243
 (inlined by) ?? at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:1486
 (inlined by) operator()<seastar::future_state<> > at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:1069
 (inlined by) run_and_dispose at /var/ssd/ceph/build/../src/seastar/include/seastar/core/future.hh:425
seastar::reactor::run_tasks(seastar::reactor::task_queue&) at /var/ssd/ceph/build/../src/seastar/src/core/reactor.cc:3590
seastar::reactor::run_some_tasks() [clone .part.0] at /var/ssd/ceph/build/../src/seastar/src/core/reactor.cc:4015
seastar::reactor::run() at /var/ssd/ceph/build/../src/seastar/src/core/reactor.cc:3998
 (inlined by) seastar::reactor::run() at /var/ssd/ceph/build/../src/seastar/src/core/reactor.cc:4158
seastar::app_template::run_deprecated(int, char**, std::function<void ()>&&) at /var/ssd/ceph/build/../src/seastar/src/core/app-template.cc:186
main at /var/ssd/ceph/build/../src/crimson/osd/main.cc:95

reproduce steps

MDS=0 MGR=1 OSD=8 MON=1 ../src/vstart.sh -n --without-dashboard --memstore -X
bin/ceph osd pool create test-pool 512 512
bin/ceph osd pool set test-pool min_size 1
bin/ceph osd pool set test-pool size 1
bin/rados bench -p test-pool 2 write --no-cleanup
for cpu in `seq 0 8`; do
  bin/init-ceph stop osd.2
  bin/crimson-osd -i $cpu -c ceph.conf --smp 1 --cpuset $cpu &
done

History

#1 Updated by Kefu Chai over 4 years ago

  • Status changed from New to Can't reproduce

not reproducible anymore.

Also available in: Atom PDF