Ceph » Messengers

[ RUN ] Messenger/MessengerTest.ReconnectRaceTest/0
2019-08-01 17:20:08.029 7f9e3f933d00 1 ceph_test_msgr SetUp start set up async+posix
2019-08-01 17:20:08.033 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) intercept called on step=1
2019-08-01 17:20:08.033 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) resuming step=1 with decision=0
2019-08-01 17:20:08.033 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=2
2019-08-01 17:20:08.033 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) resuming step=2 with decision=0
2019-08-01 17:20:08.033 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) intercept called on step=3
2019-08-01 17:20:08.033 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=4
2019-08-01 17:20:08.033 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) resuming step=3 with decision=0
2019-08-01 17:20:08.033 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) resuming step=4 with decision=0
2019-08-01 17:20:08.033 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) intercept called on step=5
2019-08-01 17:20:08.033 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) resuming step=5 with decision=0
2019-08-01 17:20:08.033 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) intercept called on step=7
2019-08-01 17:20:08.033 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) resuming step=7 with decision=0
2019-08-01 17:20:08.033 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=6
2019-08-01 17:20:08.033 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) resuming step=6 with decision=0
2019-08-01 17:20:08.033 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=8
2019-08-01 17:20:08.033 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) resuming step=8 with decision=0
2019-08-01 17:20:08.033 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) intercept called on step=9
2019-08-01 17:20:08.033 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) resuming step=9 with decision=0
2019-08-01 17:20:08.034 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=10
2019-08-01 17:20:08.034 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) resuming step=10 with decision=0
2019-08-01 17:20:08.034 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) intercept called on step=11
2019-08-01 17:20:08.034 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) resuming step=11 with decision=0
2019-08-01 17:20:08.034 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=12
2019-08-01 17:20:08.034 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) resuming step=12 with decision=0
2019-08-01 17:20:08.034 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=15
2019-08-01 17:20:08.034 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) resuming step=15 with decision=0
2019-08-01 17:20:08.034 7f9e2a662700 -1 ceph_test_msgr ms_handle_fast_connect 0x560aa523d800
2019-08-01 17:20:08.034 7f9e2a662700 -1 ceph_test_msgr ms_handle_fast_connect con: 0x560aa523d800 count: 0
2019-08-01 17:20:08.034 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) intercept called on step=15
2019-08-01 17:20:08.034 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) resuming step=15 with decision=0
2019-08-01 17:20:08.035 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=16
2019-08-01 17:20:08.035 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) resuming step=16 with decision=0
2019-08-01 17:20:08.035 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=17
2019-08-01 17:20:08.035 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) resuming step=17 with decision=0
2019-08-01 17:20:08.035 7f9e2ba64700 -1 ceph_test_msgr ms_fast_dispatch conn: 0x560aa5eea000 session 0x560aa5efa300 count: 1
2019-08-01 17:20:08.035 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) intercept called on step=16
2019-08-01 17:20:08.035 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) resuming step=16 with decision=0
2019-08-01 17:20:08.035 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) intercept called on step=17
2019-08-01 17:20:08.035 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) resuming step=17 with decision=0
2019-08-01 17:20:08.035 7f9e2a662700 -1 ceph_test_msgr ms_fast_dispatch conn: 0x560aa523d800 session 0x560aa5efa4e0 count: 1
2019-08-01 17:20:08.035 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=16
2019-08-01 17:20:08.035 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) resuming step=16 with decision=0
2019-08-01 17:20:08.035 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=17
2019-08-01 17:20:08.035 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) resuming step=17 with decision=0
2019-08-01 17:20:08.035 7f9e2ba64700 -1 ceph_test_msgr ms_fast_dispatch conn: 0x560aa5eea000 session 0x560aa5efa300 count: 2
2019-08-01 17:20:08.035 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) intercept called on step=16
2019-08-01 17:20:08.035 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) pausing on step=16
2019-08-01 17:20:08.035 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) resuming step=16 with decision=1
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) intercept called on step=1
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) resuming step=1 with decision=0
2019-08-01 17:20:08.036 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=1
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) intercept called on step=3
2019-08-01 17:20:08.036 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) resuming step=1 with decision=0
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) resuming step=3 with decision=0
2019-08-01 17:20:08.036 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=3
2019-08-01 17:20:08.036 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) resuming step=3 with decision=0
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa5eea400) intercept called on step=2
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa5eea400) resuming step=2 with decision=0
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa5eea400) intercept called on step=4
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa5eea400) resuming step=4 with decision=0
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa5eea400) intercept called on step=6
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa5eea400) resuming step=6 with decision=0
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa5eea400) intercept called on step=8
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa5eea400) resuming step=8 with decision=0
2019-08-01 17:20:08.036 7f9e2b063700 -1 ceph_test_msgr intercept conn(0x560aa5eea800) intercept called on step=2
2019-08-01 17:20:08.036 7f9e2b063700 -1 ceph_test_msgr intercept conn(0x560aa5eea800) resuming step=2 with decision=0
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) intercept called on step=5
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) resuming step=5 with decision=0
2019-08-01 17:20:08.036 7f9e2b063700 -1 ceph_test_msgr intercept conn(0x560aa5eea800) intercept called on step=4
2019-08-01 17:20:08.036 7f9e2b063700 -1 ceph_test_msgr intercept conn(0x560aa5eea800) resuming step=4 with decision=0
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) intercept called on step=7
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) resuming step=7 with decision=0
2019-08-01 17:20:08.036 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=5
2019-08-01 17:20:08.036 7f9e2b063700 -1 ceph_test_msgr intercept conn(0x560aa5eea800) intercept called on step=6
2019-08-01 17:20:08.036 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) resuming step=5 with decision=0
2019-08-01 17:20:08.036 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=7
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) intercept called on step=9
2019-08-01 17:20:08.036 7f9e2b063700 -1 ceph_test_msgr intercept conn(0x560aa5eea800) resuming step=6 with decision=0
2019-08-01 17:20:08.036 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) resuming step=7 with decision=0
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) resuming step=9 with decision=0
2019-08-01 17:20:08.036 7f9e2b063700 -1 ceph_test_msgr intercept conn(0x560aa5eea800) intercept called on step=8
2019-08-01 17:20:08.036 7f9e2b063700 -1 ceph_test_msgr intercept conn(0x560aa5eea800) resuming step=8 with decision=0
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa5eea400) intercept called on step=10
2019-08-01 17:20:08.036 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa5eea400) resuming step=10 with decision=0
2019-08-01 17:20:08.036 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=9
2019-08-01 17:20:08.036 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) resuming step=9 with decision=0
2019-08-01 17:20:08.037 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) intercept called on step=13
2019-08-01 17:20:08.037 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) pausing on step=13
2019-08-01 17:20:08.037 7f9e2b063700 -1 ceph_test_msgr intercept conn(0x560aa5eea800) intercept called on step=10
2019-08-01 17:20:08.037 7f9e2b063700 -1 ceph_test_msgr intercept conn(0x560aa5eea800) resuming step=10 with decision=0
2019-08-01 17:20:08.037 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=13
2019-08-01 17:20:08.037 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) pausing on step=13
2019-08-01 17:20:08.037 7f9e2a662700 -1 ceph_test_msgr intercept conn(0x560aa523d800) resuming step=13 with decision=0
2019-08-01 17:20:08.037 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) resuming step=13 with decision=0
2019-08-01 17:20:08.037 7f9e29c61700 -1 ceph_test_msgr ms_handle_reset 0x560aa5eea400
2019-08-01 17:20:08.038 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=13
2019-08-01 17:20:08.038 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) resuming step=13 with decision=0
2019-08-01 17:20:08.038 7f9e2885f700 -1 ceph_test_msgr ms_handle_reset 0x560aa5eea800
2019-08-01 17:20:08.038 7f9e2b063700 -1 ceph_test_msgr intercept conn(0x560aa523d800) intercept called on step=14
2019-08-01 17:20:08.038 7f9e2b063700 -1 ceph_test_msgr intercept conn(0x560aa523d800) resuming step=14 with decision=0
2019-08-01 17:20:08.038 7f9e2ba64700 -1 ceph_test_msgr ms_handle_fast_connect 0x560aa5eea000
2019-08-01 17:20:08.038 7f9e2b063700 -1 ceph_test_msgr intercept conn(0x560aa523d800) intercept called on step=15
2019-08-01 17:20:08.038 7f9e2b063700 -1 ceph_test_msgr intercept conn(0x560aa523d800) resuming step=15 with decision=0
2019-08-01 17:20:08.038 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) intercept called on step=15
2019-08-01 17:20:08.038 7f9e2ba64700 -1 ceph_test_msgr intercept conn(0x560aa5eea000) pausing on step=15
/root/work/ceph_nautilus/src/test/msgr/test_msgr.cc:794: Failure
Expected: cli_interceptor>count_step(c2s.get(), 18)
Which is: 0
To be equal to: 1u
Which is: 1
/root/work/ceph_nautilus/src/msg/async/AsyncMessenger.cc: In function 'virtual AsyncMessenger::~AsyncMessenger()' thread 7f9e3f933d00 time 2019-08-01 17:20:08.039943
/root/work/ceph_nautilus/src/msg/async/AsyncMessenger.cc: 312: FAILED ceph_assert(!did_bind)
ceph version 555a94 (d555a9489eb35f84f2e1ef49b77e19da9d113972) nautilus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1aa) [0x7f9e34777256]
2: (()+0x12904d8) [0x7f9e347774d8]
3: (AsyncMessenger::~AsyncMessenger()+0x81) [0x7f9e34ac5303]
4: (AsyncMessenger::~AsyncMessenger()+0x18) [0x7f9e34ac54ae]
5: (MessengerTest::TearDown()+0x22f) [0x560aa3f80143]
6: (void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x65) [0x560aa3ff0f15]
7: (void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x5a) [0x560aa3feb0c9]
8: (testing::Test::Run()+0x11b) [0x560aa3fd0139]
9: (testing::TestInfo::Run()+0x104) [0x560aa3fd09c6]
10: (testing::TestCase::Run()+0x107) [0x560aa3fd105f]
11: (testing::internal::UnitTestImpl::RunAllTests()+0x2d2) [0x560aa3fd7b44]
12: (bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*)+0x65) [0x560aa3ff20f7]
13: (bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*)+0x5a) [0x560aa3febfbf]
14: (testing::UnitTest::Run()+0xc9) [0x560aa3fd673b]
15: (RUN_ALL_TESTS()+0x11) [0x560aa3f7f3f2]
16: (main()+0x55d) [0x560aa3f79a55]
17: (__libc_start_main()+0xf5) [0x7f9e30a393d5]
18: (()+0xe8cd5) [0x560aa3f5fcd5]

• Caught signal (Aborted) **
  in thread 7f9e3f933d00 thread_name:ceph_test_msgr
  2019-08-01 17:20:08.046 7f9e3f933d00 -1 /root/work/ceph_nautilus/src/msg/async/AsyncMessenger.cc: In function 'virtual AsyncMessenger::~AsyncMessenger()' thread 7f9e3f933d00 time 2019-08-01 17:20:08.039943
  /root/work/ceph_nautilus/src/msg/async/AsyncMessenger.cc: 312: FAILED ceph_assert(!did_bind)