Bug #16955
Updated by Kefu Chai over 7 years ago
<pre> 2016-08-08 18:59:26.771624 7f9a65d5fd80 -1 ceph_test_msgr SetUp start set up simple 2016-08-08 18:59:26.771819 7f9a65d5fd80 -1 ceph_test_msgr ms_handle_fast_connect0xabf71a0 2016-08-08 18:59:26.771829 7f9a65d5fd80 -1 ceph_test_msgr ms_handle_fast_connect con: 0xabf71a0 count: 0 2016-08-08 18:59:26.772100 7f9a65d56700 -1 ceph_test_msgr ms_handle_fast_connect0xabf71a0 2016-08-08 18:59:26.772211 7f9a65d56700 -1 ceph_test_msgr ms_handle_fast_connect0xac8d220 2016-08-08 18:59:26.772251 7f9a65d56700 -1 ceph_test_msgr ms_handle_fast_connect con: 0xac8d220 count: 0 2016-08-08 18:59:26.772328 7f9a5c6a2700 -1 ceph_test_msgr ms_fast_dispatch conn: 0x7f9a1c001c50 session 0x7f9a20001e70 count: 1 2016-08-08 18:59:26.772337 7f9a5c6a2700 0 ------------------------------------ existing dependency SimpleMessenger::Pipe::pipe_lock (18) -> FakeDispatcher::lock (17) at: ceph version Development (no_version) 1: (Mutex::_will_lock()+0x3b) [0xf74d31] 2: (Mutex::Lock(bool)+0x50) [0xf74aac] 3: (Mutex::Locker::Locker(Mutex&)+0x2f) [0xf1035d] 4: (FakeDispatcher::ms_handle_fast_accept(Connection*)+0x39) [0xf146e9] 5: (Messenger::ms_deliver_handle_fast_accept(Connection*)+0x83) [0x1320f1b] 6: (Pipe::accept()+0x4f6a) [0x130c8a0] 7: (Pipe::reader()+0x62) [0x1316f2c] 8: (Pipe::Reader::entry()+0x1c) [0x1320f9a] 9: (Thread::entry_wrapper()+0xc1) [0x11770af] 10: (Thread::_entry_func(void*)+0x18) [0x1176fe4] 11: (()+0x7464) [0x7f1c7a9fe464] 12: (clone()+0x6d) [0x7f1c7713b30d] 2016-08-08 18:59:26.779860 7f9a5c6a2700 0 new dependency FakeDispatcher::lock (17) -> SimpleMessenger::Pipe::pipe_lock (18) creates a cycle at ceph version Development (no_version) 1: (Mutex::_will_lock()+0x3b) [0xf74d31] 2: (Mutex::Lock(bool)+0x50) [0xf74aac] 3: (SimpleMessenger::submit_message(Message*, PipeConnection*, entity_addr_t const&, int, bool)+0x454) [0x10652a4] 4: (SimpleMessenger::_send_message(Message*, Connection*)+0x2c9) [0x1062c33] 5: (SimpleMessenger::send_message(Message*, Connection*)+0x2b) [0x1326b8d] 6: (PipeConnection::send_message(Message*)+0x5d) [0x1326983] 7: (FakeDispatcher::reply_message(Message*)+0x56) [0xf154ee] 8: (FakeDispatcher::ms_fast_dispatch(Message*)+0x35a) [0xf1532e] 9: (Messenger::ms_fast_dispatch(Message*)+0xd1) [0x12f4171] 10: (DispatchQueue::fast_dispatch(Message*)+0x3e) [0x12f237e] 11: (Pipe::reader()+0x1e15) [0x1318cdf] 12: (Pipe::Reader::entry()+0x1c) [0x1320f9a] 13: (Thread::entry_wrapper()+0xc1) [0x11770af] 14: (Thread::_entry_func(void*)+0x18) [0x1176fe4] 15: (()+0x7464) [0x7f1c7a9fe464] [0x7f9a65505464] 16: (clone()+0x6d) [0x7f1c7713b30d] [0x7f9a61c4230d] 2016-08-08 18:59:26.779902 7f9a5c6a2700 0 btw, i am holding these locks: 2016-08-08 18:59:26.779903 7f9a5c6a2700 0 FakeDispatcher::lock (17) 2016-08-08 18:59:26.779904 7f9a5c6a2700 0 FakeDispatcher::Session::lock (19) 2016-08-08 18:59:26.779905 7f9a5c6a2700 0 /var/ceph/ceph/src/common/lockdep.cc: In function 'int lockdep_will_lock(const char*, int, bool)' thread 7f9a5c6a2700 time 2016-08-08 18:59:26.779915 /var/ceph/ceph/src/common/lockdep.cc: 330: FAILED assert(0) ceph version Development (no_version) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x95) [0xf82434] 2: (lockdep_will_lock(char const*, int, bool)+0xb65) [0x11b5b4f] 3: (Mutex::_will_lock()+0x3b) [0xf74d31] 4: (Mutex::Lock(bool)+0x50) [0xf74aac] 5: (SimpleMessenger::submit_message(Message*, PipeConnection*, entity_addr_t const&, int, bool)+0x454) [0x10652a4] 6: (SimpleMessenger::_send_message(Message*, Connection*)+0x2c9) [0x1062c33] 7: (SimpleMessenger::send_message(Message*, Connection*)+0x2b) [0x1326b8d] 8: (PipeConnection::send_message(Message*)+0x5d) [0x1326983] 9: (FakeDispatcher::reply_message(Message*)+0x56) [0xf154ee] 10: (FakeDispatcher::ms_fast_dispatch(Message*)+0x35a) [0xf1532e] 11: (Messenger::ms_fast_dispatch(Message*)+0xd1) [0x12f4171] 12: (DispatchQueue::fast_dispatch(Message*)+0x3e) [0x12f237e] 13: (Pipe::reader()+0x1e15) [0x1318cdf] 14: (Pipe::Reader::entry()+0x1c) [0x1320f9a] 15: (Thread::entry_wrapper()+0xc1) [0x11770af] 16: (Thread::_entry_func(void*)+0x18) [0x1176fe4] 17: (()+0x7464) [0x7f9a65505464] 18: (clone()+0x6d) [0x7f9a61c4230d] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. 2016-08-08 18:59:26.787738 7f9a5c6a2700 -1 /var/ceph/ceph/src/common/lockdep.cc: In function 'int lockdep_will_lock(const char*, int, bool)' thread 7f9a5c6a2700 time 2016-08-08 18: 59:26.779915 /var/ceph/ceph/src/common/lockdep.cc: 330: FAILED assert(0) </pre>