Ceph

9> 2016-07-16 08:00:17.102423 7f95863ac700 1 - 172.21.15.66:6805/28647 learned_addr learned my addr 172.21.15.66:6805/28647
8> 2016-07-16 08:00:17.102515 7f95863ac700 10 osd.4 5 OSD::ms_get_authorizer type=osd
-7> 2016-07-16 08:00:17.102822 7f95863ac700 1 - 172.21.15.66:6805/28647 >> 172.21.15.4:6805/2603 conn(0xd620800 sd=62 :-1 s=STATE_CONNECTING_WAIT_CONNECT_REPLY_AUTH pgs=0 cs=0 l=0).handle_connect_reply connect got WAIT (connection race)
6> 2016-07-16 08:00:17.102849 7f95863ac700 1 - 172.21.15.66:6805/28647 >> 172.21.15.4:6805/2603 conn(0xd620800 sd=62 :-1 s=STATE_WAIT pgs=0 cs=0 l=0).process enter wait state, failing
5> 2016-07-16 08:00:17.102847 7f9585bab700 10 osd.4 5 new session 0xd646ea0 con=0xd656000 addr=172.21.15.4:6805/2603
-4> 2016-07-16 08:00:17.103267 7f9585bab700 10 osd.4 5 session 0xd646ea0 osd.1 has caps osdcap[grant()] 'allow *'
-3> 2016-07-16 08:00:17.103317 7f9585bab700 0 - 172.21.15.66:6805/28647 >> 172.21.15.4:6805/2603 conn(0xd656000 sd=63 :6805 s=STATE_ACCEPTING_WAIT_CONNECT_MSG_AUTH pgs=0 cs=0 l=0).handle_connect_msg accept connect_seq 0 vs existing csq=0 existing_state=STATE_CONNECTING
2> 2016-07-16 08:00:17.103356 7f9585bab700 1 - 172.21.15.66:6805/28647 >> 172.21.15.4:6805/2603 conn(0xd656000 sd=-1 :6805 s=STATE_ACCEPTING_WAIT_CONNECT_MSG_AUTH pgs=0 cs=0 l=0)._stop
-1> 2016-07-16 08:00:17.103379 7f9585bab700 0
recursive lock of AsyncConnection::write_lock (41)
ceph version v11.0.0-632-g57264b9 (57264b9bf31cf6f77974fd7025775e963b0349e3)
1: (Mutex::_will_lock()+0x3b) [0x201adc7]
2: (Mutex::Lock(bool)+0x41) [0x201ab5d]
3: (Mutex::Locker::Locker(Mutex&)+0x2f) [0x1856bf7]
4: (AsyncConnection::_stop()+0x1c3) [0x22e5d1d]
5: (AsyncConnection::handle_connect_msg(ceph_msg_connect&, ceph::buffer::list&, ceph::buffer::list&)+0x2d7e) [0x22e0fc0]
6: (AsyncConnection::_process_connection()+0x54f5) [0x22dba31]
7: (AsyncConnection::process()+0x52af) [0x22d6093]
8: (C_handle_read::do_request(int)+0x27) [0x22ec2d3]
9: (EventCenter::process_events(int)+0x5eb) [0x212e42d]
10: (Worker::entry()+0x445) [0x211ee65]
11: (Thread::entry_wrapper()+0xc1) [0x219a749]
12: (Thread::_entry_func(void)+0x18) [0x219a67e]
13: (()+0x8182) [0x7f9599b39182]
14: (clone()+0x6d) [0x7f95986c347d]

sjust@teuthology:/a/samuelj-2016-07-14_14:34:22-rados-wip-ec-pg-log-distro-basic-smithi/314523/remote$